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Inferring  Ignorance  from  the  Locality  of  Visual  Perception 

Ernesi  Davis 

ABSTRACT 

It  is  often  possible  to  infer  that  a  certain  agent  does  not  know  about  a 
certain  occurrence,  because  he  has  had  no  way  to  find  out  about  it.  We 
present  a  formal  model  of  knowledge  acquisition  through  visual  perception 
that  supports  inferences  of  this  kind.  The  chief  innovation  of  this  model  is 
that  it  distinguishes  between  physically  possible  states  of  the  world  and 
epistemicallv  possible  worlds.  Such  a  model  allows  us  to  state  the  following 
axiom:  If  ^  »  urse  of  events  is  physically  possible,  and  an  agent's  perceptions 
are  consistent  with  that  course  of  events,  then  the  agent  cannot  know  that 
these  events  did  not,  in  fact,  occur.  We  show  how  this  axiom  supports  the 
desired  inferences  in  a  simple  models  of  physics  and  vision.  Our  model  is  rich 
enough  to  support  as  well  a  variety  of  positive  inferences,  in  which  an  agent 
gains  knowledge  of  various  kinds  through  his  perception. 
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1.    Introduction 

Intelligent  creatures  learn  much  of  what  they  know  through  direct  perception.  There- 
fore, reasoning  about  the  acquisition  of  knowledge  over  time  often  requires  a  high-level 
understanding  of  the  power  and  limits  of  perception.  This  paper  presents  a  logical  theory  that 
supports  high-level  reasoning  about  knowledge  and  perception.  We  construct  a  formal 
language  in  which  perception  can  be  described.  Using  this  language,  we  state  some  funda- 
mental axioms,  and  we  show  that  these  are  sufficient  to  justify  some  elementary  but  interest- 
ing inferences  about  perceptions. 

One  particular  tvpe  of  reasoning  that  may  be  supported  by  a  theory  of  perception  is  the 
inference  that  an  agent  must  be  ignorant  ot  a  particular  fact  because  he  has  had  no  way  to 
find  out  whether  it  is  true.  Such  means  of  inferring  ignorance  may  be  important,  either  to 
infer  that  a  secret  may  be  safeh  kept  from  another  part>',  or  to  guide  an  agent  in  planning  a 
method  to  find  out  a  given  fact.  Previous  theories  of  knowledge  have  not  supported  any 
strong  methods  of  inferring  ignorance,  other  than  a  backward  use  of  consequential  closure:  If 
A  does  not  know  Q  but  A  does  know  that  P  implies  Q  ,  then  A  must  not  know  P  .  A  theory 
of  knowledge  can  thus  be  made  a  much  more  powerful  tool  when  it  is  associated  with  a 
theory  of  perception. 

We  will  focus  on  the  following  problem: 

I.  Steve  is  in  a  closed  room  with  no  windows,  and  he  crosses  from  one  side  of  the  room 
to  the  other.  Claire  is  outside  the  room.  Infer  that  Claire  does  not  know  now  that  Steve  has 
crossed  the  room. 

The  above  problem  is  a  somewhat  dangerous  benchmark  for  a  theory,  since  it  is  trivially 
satisfied  in  a  theory  in  which  no  one  knows  anvthing.  To  ensure  that  our  theory  is  not  trivial 
in  this  sense,  we  require  that  it  support  the  following  inferences  as  well: 

II.  Andrew  is  in  his  office  and  he  does  not  see  any  cows  there.  Infer  that  he  knows  that 
there  are  no  cows  in  his  office. 

III.  Joanne,  in  the  living  room,  does  not  know  whether  there  are  flowers  on  the  dining 
room  table.  However,  she  knows  that  the  top  of  the  dining  room  table  is  visible  from  any 
point  in  the  dining  room.  Infer  that  Joanne  knows  that  she  can  find  out  whether  there  are 
flowers  on  the  dining  room  table  by  going  and  looking. 

IV.  Fred  has  seen  that  .Max  has  been  with  him  all  night,  playing  poker.  Infer  that  Fred 
knows  that  Max  was  not  five  miles  away,  at  the  bank  robbery. 

V.  Judy  sees  Sharon  standing  facing  a  bus.  and  looking  at  the  bus.  Infer  that  Judy 
knows  that  Sharon  knows  that  there  is  a  bus  in  front  of  her. 

These  problems  illustrate  various  key  aspects  of  perception.  (I)  shows  how  we  can  infer 
that  an  agent  is  ignorant  of  a  fact  from  our  knowledge  of  the  physical  limits  of  vision.  Since 
Claire  cannot  see  Steve  inside  the  room,  she  cannot  know  what  is  happening  in  the  room. 
(II)  shows  the  gaining  of  knowledge  from  inferences  based  on  both  prior  knowledge  and  per- 
ception.   Andrew  infers  the  absence  of  cows  from  the  fact  that,  given  what  he  knows  about 
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the  size  of  cows,  the  presence  of  a  cow  is  physically  incompatible  with  what  he  sees.  (Ill) 
involves  reasoning  about  possible  future  states  of  perception  and  knowledge.  (IV)  involves 
perception  extended  over  an  interval  of  time.  (V)  shows  that  one  agent  can  infer  the  percep- 
tions and  knowledge  of  another  agent  by  perceiving  their  physical  situations  and  knowing 
their  perceptual  and  inferential  powers.  The  theory  developed  in  this  paper  supports  infer- 
ences of  all  these  types. 

The  formal  model  developed  in  this  paper  is  rich  enough  to  support  the  statement  and 
proof  of  close  analogues  of  all  the  about  inferences.  Section  2  gives  an  informal  characteriza- 
tion of  the  world  described  by  the  model.  Section  3  describes  the  formal  model,  a  variant  on 
the  well-known  possible  world  semantics  for  knowledge  [Hintikka.  71],  [Moore,  80].  Also,  in 
section  3.  we  show  that  our  model  implies  a  limited  version  of  negative  introspection:  If  an 
agent  does  not  know  some  fact  about  the  physical  course  of  events,  then  he  knows  that  he 
does  not  know  it.  Section  4  presents  analogues  of  the  above  inferences,  and  sketches  their 
proofs.  Section  5  discusses  some  difficulties  with  the  model,  and  section  6  presents  our  con- 
clusions. The  appendix  gives  the  complete  proof  of  an  inference  analogous  to  example  (I). 

It  should  be  noted  that  our  purposes  are  quite  different  from  those  of  computer  vision 
research;  we  therefore  need  a  theory  of  perception  at  a  verv  different  level  of  description. 
There  is  no  computer  vision  program  that  represents  or  uses  the  facts  about  vision  needed  to 
solve  problems  (I)  through  (V)  above.  For  example,  consider  the  fact  that  it  is  impossible  to 
see  what  is  happening  outside  a  closed  room  from  inside  it.  or  the  fact  that  it  is  hard  to  see 
from  a  brightly  lit  area  into  a  dark  area.  Such  facts  are  of  little  importance  in  computer 
vision,  since  they  very  rarely  offer  a  significant  constraint  on  the  interpretation  of  a  particular 
image.  However,  they  are  of  great  importance  in  reasoning  qualitatively  about  how  vision 
will  augment  knowledge  in  a  given  situation;  one  can  use  them  to  deduce  that  an  agent  will 
have  to  leave  a  closed  room  to  know  what  is  happening  outside  it,  or  that  he  will  have  to  turn 
olf  the  lights  in  his  car  to  know  what  is  on  the  highway.  Conversely,  our  theory  need  not 
deal  with  the  specific  rules  and  algorithms  necessary  to  interpret  an  actual  image,  because 
you  do  not  generally  have  to  interpret  any  images  other  than  the  one  before  you,  which  is 
tackled  by  the  vision  iystem.  For  exampiC,  the  rule  that  a  large  change  in  detected  light  inten- 
sity may  indicate  a  significant  edge  in  an  image  is  a  key  fact  in  image  interpretation,  but  is  of 
no  value  in  inference  about  future  or  hypothetical  state,  since  one  cannot  usually  predict  the 
light  intensities  to  be  detected. 

.A  few  limitations  of  our  theory  should  be  noted  at  the  outset.  First,  it  ignores  learning 
through  perceptions  of  conventional  signs  such  as  speech  and  writing.  Perceptions  are  only 
used  for  their  direct  physical  implications.  Second,  we  assume  that,  under  proper  conditions. 
It  can  be  known  that  a  particular  event  has  only  limited  physical  effects.  Third,  we  model 
only  visual  perception.  We  believe,  however,  that  other  forms  of  perception  can  be  treated 
similarly.  Finally,  our  models  of  vision  and  of  physical  causality  are  greatly  simplified.  We 
are  interested  here  in  the  connection  between  perception  and  knowledge,  and  a  more  com- 
plex physical  theory  would  just  have  added  irrelevant  complexity.  We  believe  that  the  essen- 
tial structure  of  our  theory  will  carry  over  to  more  realistic  physical  models. 

2.    The  Toy  World 

We  formulate  our  theory  within  a  highly  simplified  "toy"  world,  which  we  describe 
informally  in  this  section.  Many  of  the  particular  assumptions  made  below  are  obviously  not 
valid  in  general;  however,  the  basic  structure  does  not  depend  strongly  on  these  assumptions. 
In  particular,  the  relations  between  knowledge  and  perception,  which  are  our  focus  here,  are 
largely  independent  of  the  particular  physics  of  motion  and  vision  assumed. 

Our  toy  world  is  constructed  as  follows:  There  is  a  fixed  set  of  physical  objects  scattered 
in  space,  moving  about  over  time.  Objects  are  rigid,  maintaining  a  constant  shape;  however, 
their  position  may  change  continuously  over  time.  At  each  time  instant,  each  object  occupies 
a  figure  ;  a  connected  regular'  set  of  points.  The  places  occupied  by  two  objects  at  a  single 


i.  A  set  IS  regular  if  it  is  closed,  bounded,  and  equal  to  the  closure  of  its  interior. 


-  3 


time  mav  not  overlap.  Besides  their  shapes,  objects  have  time-invariant  visible  properties. 
such  as  their  coloring.  Objects  may  be  also  be  characterized  in  terms  of  non-physical  proper- 
ties and  relations,  such  as  "being  a  Republican",  or  "being  brothers." 

There  are  no  other  physical  restrictions  on  the  world,  beyond  those  mentioned  above.  A 
course  of  events  is  physically  possible  if  and  only  if  each  object  maintains  a  constant  shape 
and  properties  and  moves  continuously,  and  no  two  objects  ever  overlap. 

Some  objects  are  agents.  At  each  instant  of  time,  an  agent  has  a  body  of  knowledge 
with  the  following  properties: 

A.l    Knowledge  of  axioms:  All  general  axioms  —   axioms  of  predicate  calculus,  geometry, 
time,  physics,  knowledge  and  perception  —  are  known. 

A. 2    Consequential  closure:  Any  logical  implication  of  the  agent's  knowledge  is  known. 

A. 3    Veridicality:  All  knowledge  is  true. 

.■\.4    Positive  introspection:  If  an  agent  knows  a  fact,  he  knows  that  he  knows  it. 

A. 5    Memory:  If  an  agent  knows  a  fact  (with  no  time  indexicals)  at  one  time,  he  knows  it  at 
all  later  times. 

A. 6    Internal  Clock:  An  agent  always  knows  what  time  it  is.  (This  assumption  is  not  strictly 
necessary,  but  it  makes  things  easier.) 

.•\n  agent  also  has  perceptions.  A  point  is  visible  to  an  agent  if  it  is  not  occluded  from 
him  by  an  object  m  between.  (An  obstacle  O  occludes  a  point  P  from  an  agent  A  if  any  line 
segment  from  A  w  P  intersects  O  (Figure  1).)  All  objects  are  thus  assumed  to  be  opaque.  An 
agent  A  can  see  tne  visible  properties  of  an  object  at  a  visible  point.  .Moreover,  A  can  see  the 
properties,  not  only  of  individual  points  but  also  of  connected  sets  of  points.  Specifically,  let 
X  be  a  connected  set  such  that  each  point  of  X  is  visible  to  A  .  U  all  of  X  lies  inside  an  object 
O  with  visible  properties  PI,  P2.  ...,  then  A  sees  that  A'  lies  in  some  object  with  properties 
P\.P2,...    If  X  lies  in  free  space,  then  4  sees  that  A'  lies  entirely  in  free  space. 

Perception  provides  knowledge  about  the  world,  and  it  is  the  only  source  of  information 
as  to  which  of  the  many  physically  possible  courses  of  events  is  actually  happening.  Specifi- 
cally, we  assume  the  following: 

.A. 7    Anything  that  is  perceived  is  known. 

A. 8    If  a  physical  statement  is  physically   possible,   and  it  does  not  contradict  any  past  or 
present  perceptions,  then  it  cannot  be  known  to  be  false. 

As  we  shall  see  in  section  4.  this  model  is  rich  enough  to  allow  us  to  pose  reasonable 
analogues  of  problems  I  through  V  above. 

3.    Formal  Model 

Our  model  of  knowledge  derives  from  Hintikka's  (1971)  possible  worlds  semantics  for 
knowledge.  Moore  (1980)  combined  this  with  a  temporal  logic,  by  identifying  epistemic  pos- 
sible worlds  and  temporal  situations.  We  modify  Moore's  model  in  two  ways.  First,  we  adopt 
a  continuous  rather  than  a  discrete  model  of  time  [McDermott.  82].  Second,  and  more  impor- 
tantly, we  use  two  levels  of  possible  worlds:  layouts  and  situations.  A  layout  is  a  timeless 
physical  description  of  the  instantaneous  state  of  the  world.  A  situation  is  a  placement  of  a 
layout  within  a  temporal  structure  and  within  a  system  of  knowledge  relations.  Perceptions 
are  associated  with  layouts;  knowledge  is  associated  with  situations. 

In  our  toy  physics,  an  object  is  an  atomic  individual  with  a  set  of  visual  properties.  A 
layout  specifies  the  objects  in  the  world  and  the  figure  occupied  by  each  object.  The  figure 
occupied  by  an  object  O  in  layout  L  is  denoted  "place(0  ,Z. )". 

A  physical  behavior  is  an  account  of  the  physical  layout  over  all  time.  Formally,  it  is  a 
function  from  time  to  physical  layouts  with  the  following  properties: 
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a.  All  layouts  in  the  behavior  have  the  same  objects. 

b.  Objects  have  constant  shape  throughout  the  behavior.  That  is.  for  anv  object  O, 
behavior  B.  and  times  T\  and  T2.  the  two  figures  place(0  ,scene(fi  .7"  1))  and 
place(0  .scenetfl  ,7"2))  are  congruent. 

c.  Objects  move  continuously  in  time.  That  is,  the  function  over  time  to  figures 
\(7")  place(0  .scene(S  ,T))  is  continuous,  for  any  object  O  and  behavior  B  . 

The  function  "scenefS.D"  maps  a  behavior  B  and  a  time  T  to  the  layout  of  B  at  time 
7".  Time  is  real  valued.  The  onlv  relation  that  we  will  need  on  time  is  the  total  ordering,  71 
is  earlier  than  or  equal  to  7"2,  written  71^72. 

We  distinguish  certain  layouts  and  behaviors  as  physicalh  possible.  A  layout  is  physi- 
cally possible  if  no  two  objects  overlap.  A  behavior  is  physically  possible  if  each  layout  in  the 
behavior  is  physically  possible.  We  allow  physically  impossible  layouts  and  behaviors  as 
valid  objects  of  thought;  this  simplifies  the  statement  of  the  physical  axioms.  Note  that  condi- 
tions (a),  (b),  and  (c)  above  apply  even  to  physically  impossible  behaviors. 

What  the  agent  can  see  in  a  given  physical  layout  is  determined  bv  the  physical  laws  of 
vision.  The  perceptions  of  an  agent  A  \n  a  layout  L  fix  all  aspects  of  L  at  points  which  are 
not  occluded  from  .4  ,  and  fix  no  aspects  of  L  at  points  which  are  occluded  from  A  .  We  say 
that  a  lavout  L  1  is  visually  compatible  with  L  with  reference  to  A  ,  written 
'v_compatible(A  .Z.  ,L  1)"  if  Z.  1  is  consistent  with  everything  that  A  can  perceive  in  L. 
V_compatibility  is  an  equivalence  relation  over  layouts. 

Our  theory  of  the  scope  and  limits  of  vision  is  expressed  in  terms  of  the  properties  of 
the  v_compatible  relation.  In  our  simple  physics,  Z.  2  is  v_compatible  with  L  1  with  respect  to 
agent  .4  if  both  L2  and  L  1  are  physically  possible  and  the  following  condition  holds:  Let  X  be 
a  connected  set  of  points,  such  that  every  point  in  A'  is  visible  to  A  in  L  1.  Then  each  point  of 
X  is  visible  to  A  in  L2.  Moreover,  if  X  lies  entirely  in  A  in  Z.  1,  then  X  lies  entirely  in  A  in 
L2;  if  A'  lies  entirely  in  some  object  0\  in  L\.  then,  in  Z.  2,  X  lies  entirely  inside  some  object 
02  with  the  same  visible  properties  as  01;  if  A  lies  entirely  in  tree  space  in  Z.  1  then  A  lies 
entirely  in  free  space  in  Z.2. 

A  behavior  fl  1  is  visually  compatible  with  behavior  flO  for  agent  A  up  to  time  7  if,  as 
far  as  A  can  see  in  flO  up  to  time  7,  the  world  could  be  going  through  B  1.  We  write  this 
relation,  "bv_compatible(A  ,fl  0,B  1.7)".  In  our  simple  model  of  vision,  we  will  assume  that 
all  such  information  comes  through  the  layouts;  that  is,  two  physically  possible  behaviors  are 
visually  compatible  if  corresponding  layouts  are  visually  compatible. 

bv_compatible{A  ,fi  l.fi2,75)   = 

V  -Tcj-s  v_compatible(A  ,situation(S  1.7'),situation(fi  2,7)) 

Two  states  of  the  world  may  be  identical  in  their  physical  layout  and  yet  differ  in  other 
respects.  To  accommodate  this,  we  define  a  situation  as  a  state  of  the  world,  including  the 
physical  layout,  the  non-physical  properties  and  relations  of  objects,  and  the  knowledge  states 
of  agents.  The  knowledge  states  of  agents,  however,  are  not  a  component  of  the  situation, 
but  are  encoded  in  accessibility  relations  between  situations.  Formally,  we  can  define  a  situa- 
tion as  a  triple  <Z,,<t),A>,  where  Z.  is  a  layout,  <t>  is  a  function  mapping  each  non-physical 
relation  among  objects  to  its  extension  as  a  set  of  tuples  of  objects,  and  X  is  an  atomic  indivi- 
dual, which  serves  to  distinguish  situations  that  differ  only  in  the  associated  knowledge  states. 
(The  problem  here  is  that  we  want  the  option  of  having  two  different  situations  that  are  the 
same  in  all  respects  except  the  knowledge  states.  We  accomplish  this  by  giving  them  unequal 
values  of  X .  The  entity  X  serves  no  other  function.)  The  function  "Iayout(5)"  maps  a  situa- 
tion to  its  layout. 

Non-physical  properties  of  objects  are  made  parts  of  situations  rather  than  of  layouts,  in 
order  to  allow  different  agents  to  have  different  degrees  of  knowledge  about  them.  In  our 
system,  we  can  allow  Tom  to  know  that  all  cows  are  large,  but  Sid  not  to  know  this.  This 
would  not  be  possible  if  we  associated  the  non-physical  property  of  being  a  cow  with  lavouts. 


-  D 


The  knowledge  of  an  agent  in  a  situation  S  is  represented  by  an  accessibility  relation 
berween  5  and  other  situations  that  are  consistent  with  his  knowledee.  Let  A  be  an  agent  and 
let  5  1  and  5  2  be  two  siruaiions.  S2  is  accessible  from  51  relative  to  A  .  written  "k(/4  ,5  1.5  2)", 
if  as  far  A  knows  in  5  1.  the  state  of  the  world  might  be  52.  We  sav  that  A  knows  in  5  1  that 
<i>  IS  true  if  <t>  is  true  in  all  situations  that  are  accessible  to  A   from  5  1. 

Note  that  in  both  the  visual  compatibility  relation  and  the  knowledee  accessibility  rela- 
tion, more  information  corresponds  to  a  smaller  extension  of  the  relation.  The  more  you 
know,  the  more  variations  in  the  world  you  can  rule  out  as  false,  and  therefore  the  fewer 
possible  states  of  the  world  are  consistent  with  your  knowledge. 

A  chronicle  is  a  function  from  time  to  situations.  The  function  "situationfC  ,7)"  maps  a 
chronicle  C  and  a  time  T  onto  the  situation  in  that  chronicle  at  that  time.  We  assume  that 
each  situation  5  has  a  unique  time  m  a  unique  chronicle,  denoted  "timef5)"  and 
"chronicle(5)".  (This  restriction  is  not  strictly  necessary,  but  it  simplifies  descriptions  without 
any  loss  of  generality.)  Each  chronicle  C  has  associated  a  behavior,  which  is  the  progression 
of  layouts  of  the  scenes. 

behavior(C  )   =   \(7" )  layout(situation(C  ,7" )) 

If  situations  5  1  and  5 2  are  in  the  same  chronicle,  and  the  time  of  5  1  is  earlier  than  the  time 
of52.  we  say  thai51  precedes  52. 

precedes(5  1,52)  =  [  chronicle(51)  ^   chronicle(52)  '  time(51)  s  time(52)  ] 

Knowledge  about  the  past  or  the  future  is  represented  by  combining  the  epistemic  acces- 
sibility relation  with  the  structure  of  chronicles.  ".Mark  knows  that  the  flower  was  in  the  vase 
at  9:00,"  is  represented  in  the  following  form:  "Let  5  1  be  a  situation  accessible  relative  to 
.Mark  from  the  current  situation  sO.  Let  C  be  the  chronicle  of  5  1.  Let  52  be  the  situation  of 
C  at  9:00.  Then  the  place  of  the  flower  in  52  was  inside  the  place  of  the  vase  in  52."  (Figure 
4). 

V^i  5;  [  k(mark,  sO,  5  1)  ',  52  =  situation{chronicle(5  l),tyOO)  ]  =• 
inside (place( flower  1.5 2),  place(vasel,52)) 

We  can  achieve  the  properties  of  knowledge  and  perception  enumerated  above  A.l  - 
.A. 8  by  imposing  the  following  requirements: 

A.l,  knowledge  of  the  axioms,  and  A. 2,  consequential  closure,  follow  immediately 
from  the  definition  of  knowledge,  together  with  the  axiom  that  all  chronicles  have  a  physi- 
cally possible  behavior. 

V-  phys_poss(behavior(C )) 

A. 3,  veridicality.  is  satisfied  if  the  knowledge  accessibility  relation  is  reflexive. 

V,  ,  k(/l,5,5) 

A. 4,  positive  introspection,  follows  if  knowledge  accessibility  is  transitive. 
^A  5i.5:.53  ^(^4  ,5  1,52)  A  k(A  ,52,53)  ^-  k(A  ,51,53) 

A. 5,  memory,  holds  given  the  following  condition:  Let  situation  5  15  be  accessible  from 
50fi,  and  let  50,4  precede  50fi  in  the  same  chronicle.  Then,  since  everything  the  agent 
knows  in  50A  ,  he  also  knows  in  50fl,  and  515  is  consistent  with  everything  he  knows  in 
505  ,  there  must  be  a  scene  5  L4  in  the  chronicle  of  5  16  which  is  accessible  from  5  0A  .  (Fig- 
ure 5) 

^  A  5  IB  SOB  SOA  [^(A'S  05  ,5  15  )  /  precedes(5  OA  ,5  05  )  J  s- 
3  . ,  ^    [  k(A  ,5  OA  ,5  lA  )  A  precedes(5  lA  ,5  15  )  ] 
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A. 6.  internal  clock,  holds  given  the  condition  that,  if  S  1  is  accessible  from  SO.  then  the 
two  situations  occur  at  the  same  time. 

V  ,  3^,,  k(A  ,50,5  n  =■  time(50)  =  time(5  1) 

A. 7.  that  perceptions  are  known,  holds  given  the  following:  If  5  1  is  knowledge  accessi- 
ble from  5  0.  then  the  lavout  of  5  1  is  visually  compatible  with  the  layout  of  5  0.  That  is,  for  a 
situation  5  1  to  be  consistent  with  an  agent's  knowledge,  the  layout  of  5  1  must  be  compatible 
with  what  the  agent  sees;  conversely,  if  a  layout  is  not  compatible  with  what  the  agent  sees, 
the  agent  knows  that  that  cannot  be  the  real  layout. 

V  ^  50  SI  k(-4  ,50,51)  -■  v_compatible(>l  ,layout(50),layout(5  1)) 

A. 8.  that  perception  is  the  only  source  of  knowledge  of  the  course  of  events,  holds 
given  the  following:  Let  C 0  be  the  real  chronicle.  Let  B  1  be  a  behavior  that  is  visually  com- 
patible with  the  behavior  of  CO  up  to  time  7"  relative  to  agent  A  :  thus,  as  far  as  A  could  have 
seen  up  to  time  T ,  B\  could  be  the  real  behavior.  Then  it  is  consistent  with  A  's  knowledge 
that  B  1  actuallv  was  the  real  behavior;  that  is,  there  is  a  chronicle  C  1  whose  situation  in  T  is 
knowledge  accessible  from  the  situation  of  CO  in  T . 

^  A  CO  BIT  bv_compatible(A  ,behavior(CO),fl  l,r)  = 

3-    i  k(.4  .siiuation(C0.r),situation(Cl,7))   ■  B\  =   behavior(C  1)  ] 

Two  general  theorems  of  some  interest  loUow  from  the  axioms.  The  first  is  the  analo- 
gue 01  axiom  A. 7  for  perception  over  time:  Facts  perceived  over  time  are  known.  Formally, 
if  5  1  is  knowledge  accessible  from  50,  then  the  behavior  of  the  chronicle  of  50  up  50  is  visu- 
ally compatible  with  the  behavior  of  the  chronicle  of  5  1  up  to  5  1. 

V,5p_,,  k(/\  .50.51)  = 

bv_compatible(A  ,behavior(chronicle(50)),  behavior(chronicle{5  1)),  time(50)) 

The  second  theorem  is  a  restricted  form  of  /le^^ar/vf  introspection:  If  an  agent  does  not 
know  a  fact  about  the  course  of  events,  then  he  knows  that  he  doesn't  know  it.  The  formal 
statement  is  that,  if  a  behavior  is  compatible  with  an  agents  perceptions,  then,  from  any 
knowledge  accessible  situation,  there  is  a  knowledge  accessible  chronicle  exhibiting  that 
behavior.  That  is.  in  all  situations  compatible  with  what  the  agent  knows,  there  is  a  compati- 
ble chronicle  in  which  the  behavior  occurs;  i.e.  the  agent  knows  that  he  doesn't  know  that  the 
behavior  B  1  didn't  occur. 

'  A  CO  BITS'..' 

[  bv_compatible(A  .behavior(CO),S  l.D  '   k(A  .situation(C0,r).5Af  )  1  = 
3r,  [  k(A  ,5A/,situation(Cl,7))  /  bv_compatibleM  ,behavior(Cl).S  1,7)  j 

The  proofs  of  these  two  theorems  are  sketched  in  the  appendix  (Theorem  4.15  and 
4.16). 

4.    Proofs 

In  this  section  we  show  how  analogues  to  inferences  (I)  through  (V)  can  be  formulated 
in  our  model,  and  we  will  sketch  how  they  can  be  proven  from  the  above  axioms  of  percep- 
tion and  knowledge,  together  with  suitable  axioms  of  geometry  and  physics.  In  our  discussion 
we  will  show  some  of  the  logical  formulas  that  arise  in  these  proofs,  including  the  logical 
statement  of  the  conclusion  of  the  proof.  We  will  not  here  define  all  the  non-logical  symbols 
introduced.  Most  of  them  are  defined  in  the  appendix;  the  rest  can  be  easily  deciphered  using 
context  and  English  translation.  Quantified  variables  are  indicated  with  italicized  capital 
letters;  free  variables  are  assumed  to  quantified  over  the  entire  formula.  The  complete  proof 
of  example  (I)  may  be  found  in  the  appendix.  This  complete  proof  uses  a  standard  first-order 
sorted  logic,  with  equality  and  function  symbols. 


I.  Claire  is  on  one  side  of  a  wall,  lor  some  interval  of  time.  On  the  other  side  of  the 
wall,  occluded  from  Claire,  is  an  oh)eci  omystery.  The  object  lies  strictly  within  some  larger 
region,  which  is  entirely  occluded  from  Claire.  (Figure  2).  Over  a  certain  time  interval  iO. 
Claire  stavs  motionless,  the  object  stavs  within  its  envelope,  and  no  other  object  ever  inter- 
sects the  envelope.  We  wish  to  prove  that  there  is  no  way  for  Claire  to  know  whether  the 
object  is  motionless  or  whether  it  is  moving  around  within  its  envelope,  since  either  are 
equally  compatible  with  the  motions  of  the  objects  that  Claire  does  see.-  This  conclusion  can 
be  formalized  as  follows:  At  the  end  of  iO.  there  is  one  knowledge  accessible  situation  that 
follows  on  a  chronicle  in  which  the  object  is  motionless;  there  is  another  knowledge  accessible 
situation  that  follows  on  a  chronicle  in  which  the  object  is  in  motion. 

[3(;i  k(aclaire.sOz.  situationCC  l,end(iO)))  /   motionlessfomystery.behavior(C  l),iO)  ]  A 
[3(-,  k(aclaire,s0z,situation(C2.end(i0)))  /   ^motionlessComystery,behavior(C2),iO)  ] 

To  prove  this,  we  construct  two  particular  behaviors.  In  the  first,  every  object  moves 
just  as  It  does  in  the  real  world  except  that  omystery  stays  motionless  throughout  iO.  In  the 
second  every  object  moves  just  as  it  does  in  the  real  world  except  that  omystery  moves  con- 
iinuously  within  xenvelope  throughout  iO.  We  show  that  both  ot  these  are  phvsicallv  possible, 
since  no  other  object  comes  withm  the  envelope,  b\  hypothesis,  and  so  no  other  object 
interacts  with  omystery.  Both  compatible  with  Claire's  perceptions,  since  the  identical  objects 
are  visible  to  Claire  in  the  identical  places.  Hence,  by  axiom  .A. (5,  Claire  cannot  know  of 
either  ot  them  that  it  did  not  occur. 

II.  Andrew  knows  that  he  is  in  a  closed  square  room  20  feet  by  20  feet.  He  knows  that 
all  cows  contain  a  circular  disk  2  feet  in  diameter.  Andrew  is  in  the  center  of  the  room. 
Every  other  object  m  the  room  lies  entirely  within  a  foot  of  the  wall.  Thus,  from  what 
Andrew  knows  of  the  room  of  cows,  and  from  what  he  can  see  of  the  other  objects,  he  can 
infer  that  no  cow  could  fit  into  the  occluded  parts  of  the  room.  We  wish  to  prove  that 
Andrew  knows  that  there  is  no  cow  in  the  room. 

This  conclusion  can  be  formalized  as  the  statement  that  in  any  knowledge  accessible 
world,  any  cow  is  outside  the  room. 

[  k(andrew.5  0,51)  a  true_in(5  1,cow(C»COM"))  1  =■ 

-sub_place( place (OCOH'  ,layout(5  1)),  place ( inside (orooml). 5  111  | 

Note  that  we  treat  "being  a  cow"  as  a  contingent  property;  otherwise,  we  would  be  commit- 
ting to  supposing  that  everyone  would  always  know  whether  a  given  object  was  a  cow.  The 
term  "inside(orooml)"  here  designates  a  pseudo-object,  that  is,  a  point  set  that  moves  around 
with  the  room  [Davis,  86b]. 

Andrew's  knowledge  that  all  cows  contain  a  disk  two  feet  in  diameter  can  be  formalized 
as  the  statement  that,  in  all  knowledge  accessible  worlds,  all  cows  have  that  property. 

[  k(andrew ,50,51)  and  true_in(5  l,cow(OCOW  ))  )  =• 
3y  disk(A:,two_feet)  a  sub_place(X  ,place(OCOW  ,5  1)) 

This  assumption  does  not  contradict  axiom  A. 8,  that  all  information  about  the  course  of 
events  is  gathered  from  perception,  if  we  assume  that  being  a  cow  is  not  an  aspect  of  the  phy- 
sical course  of  events  and  does  not  enter  into  layouts,  but  only  into  situations.  We  will  dis- 
cuss this  point  further  in  section  5, 

We  are  given  that  Andrew  knows  lower  bounds  on  the  size  of  cows,  that  he  knows  the 
size  and  shape  of  the  room,  and  that  all  objects  in  the  room  other  than  himself  lie  within  a 
foot  of  the  walls  of  the  room.  By  a  geometrical  argument,  then,  in  any  layout  compatible 


2  The  particular  geome;rv  chosen  here  is  not.  by  any  means,  the  only  one  tha;  will  allow  the  argument 
to  go  through:  however,  ii  simpliiies  the  geometrical  analysis.  The  example  of  a  closed  room  is  too 
strong:  in  our  ihecr>  an  actor  outside  a  closed  room  can  never  find  out  anything  about  what  happens 
inside  the  room. 
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with  what  he  sees,  the  entire  center  of  the  room  is  clear  of  obiects.  (Note  that,  since  the 
obiects  partiallv  occlude  the  walls.  Andrew  cannot  at  the  moment  see  that  they  are  all  within 
a  foot  of  the  walls:  that  is.  there  are  compatible  layouts  in  which  the  walls  are  further  back.) 
Bv  axiom  A. 7,  that  perceptions  are  known,  it  follows  that  in  any  knowledge  accessible  situa- 
tion, the  entire  center  of  the  room  is  clear.  But  Andrew  knows  the  structure  of  the  room;  that 
is.  in  all  knowledge  accessible  situations,  the  walls  of  the  room  have  their  true  shape.  There- 
fore, in  all  knowledge  accessible  situations,  all  visible  objects  lie  within  a  foot  of  the  wall. 
Therefore,  bv  another  geometrical  argument,  in  every  accessible  world,  the  room  does  not 
contain  any  object  containing  a  disk  of  rwo  foot  diameter,  since  such  an  object  would  either 
overlap  the  walls  or  the  empty  center  space.  But  in  all  accessible  situations,  any  cow  is  con- 
tains a  rwo-foot  diameter  disk.  Hence,  in  all  accessible  situations,  there  is  no  cow  in  the 
room. 

III.  Joanne  is  inside  some  enclosure  with  a  cat  and  with  a  U-shaped  room.  Joanne 
knows  the  shape  of  the  room.  We  wish  to  prove  that  Joanne  knows  that,  if  she,  the  cat,  and 
the  room  are  the  only  things  within  the  enclosure,  and  she  goes  into  the  room,  then  she  will 
know  whether  the  cat  is  also  in  the  living  room  (Figure  3). 

This  conclusion  can  be  formalized  as  follows:  Let  50  be  the  real  situation.  Let  51  be 
any  situation  knowledge  accessible  from  50  such  that,  in  51,  Joanne,  the  cat,  and  the  room 
are  the  only  objects  in  the  enclosure.  Let  5  2  be  a  situation  later  than  5  1  in  which  Joanne  is  in 
the  room.  Then  one  of  two  things  are  true:  Either,  in  all  situations  accessible  from  5  2,  the 
cat  IS  in  the  room,  or,  in  all  situations  accessible  from  52.  the  cat  is  not  in  the  room. 

[  kfajoanne,50,5  1)  / 
[  Vp  sub_place(place(0  ,layout(51)),place(inside(oenclosure),layout(5  1)))  ■-■ 

[  O  -  ajoanne   /  O  =  ocat  .  O  -  oroom  |  ]  / 
precedesf5  1.52)   \ 
sub_place(place(ajoanne.lavout(5  2)).place(inside(oroom),lavout(5  2))) 

[  [  V,3  [k(ajoanne,5  2,5  3)  =- 

sub_place(place(ocat.layout(5  3)'),place(inside(oroom),layout(5  3))l  ]   . 
[  V^3  [k(ajoanne,52,53)  = 

-sub_place(piace(ocat.layout(5  3)),place(inside(oroom),layout(5  3)))  ]  ] 

Proof:  If  Joanne,  the  cat.  and  the  room  are  the  only  objects  inside  the  enclosure  in 
situation  51,  they  must  be  the  only  objects  inside  the  enclosure  in  situation  52.  since,  by  phy- 
sical laws,  nothing  can  go  through  a  closed  barrier.  Bv  a  geometrical  argument  from  the 
shape  of  the  room  we  can  show  that,  (i)  if  Joanne  is  the  only  object  in  the  room,  then  she  can 
see  the  entire  inside  area  of  the  room  as  well  as  the  inner  walls  of  the  room,  and  thus  deter- 
mine that  she  is  inside  the  room,  and  that  nothing  else  is;  (li)  if  Joanne  and  the  cat  are  both 
inside  the  room,  then  the  cat  will  be  visible  and  enough  of  the  inner  walls  of  the  room  will  be 
visible  that  Joanne  will  be  able  to  see  that  she  is  inside  the  room.  (Keep  in  mind  that  Joanne 
not  only  has  to  see  the  cat  at  a  place  that  is  in  fact  inside  the  room;  she  also  has  to  see 
enough  of  the  room  to  know  that  the  place  of  the  cat  is  inside  the  room.)  Either  case  (i)  or 
case  (ii)  must  hold.  Using  axiom  A. 7.  that  what  is  seen  is  known,  we  deduce  that,  in  case  (i) 
Joanne  will  know  that  the  cat  is  not  inside  the  room,  while  in  case  (li),  Joanne  will  know  that 
the  cat  is  inside  the  room. 

Two  limitations  of  our  theory  should  be  noted  in  connection  with  this  example.  The  first 
is  that  we  cannot  express  the  fact  that  Joanne  can  go  into  the  room  if  she  chooses  to  do  so, 
since  we  have  no  planning  vocabulary.  The  second  is  that,  due  to  the  extremely  strong 
requirement  that  facts  about  the  course  of  events  can  only  be  known  through  perception, 
there  is  no  way  that  Joanne  can  ever  find  out  that  there  are  no  other  obiects  in  the  enclosure. 
The  other  obiects  could  just  have  hidden  from  her  up  to  now.  and  be  ready  to  spring  out  and 
surround  her  when  she  tries  to  look  for  the  cat.  We  will  discuss  this  further  in  section  5. 


I\'.  Fred  knows  that  Max  has  a  diameter  less  than  seven  feet.  During  the  time  interval 
lO.  Max  was  always  within  ten  feet  of  Fred,  and  Fred  always  has  an  unoccluded  view  of  him. 
We  wish  to  prove  Fred  knows  that  .Max  was  not  one  hundred  feet  away  durmg  any  part  of  iO. 

Formally,  let  5  1  be  a  knowledge  accessible  situation,  let  C  1  be  the  chronicle  of  5  1.  and 
let  r  be  a  time  in  iO.  Then  in  the  situation  of  CI  at  7,  no  point  of  Max  is  one  hundred  feet 
from  any  point  of  Fred. 

[  k(afred,50.51)  r  C  1  =  chroniclef5  1)  a  time_in(r,iO)  /  L7"  =  layout(situation(Cl,r)) 

'>  point_in(/'F  ,pIace(afred.Z.7))  /  point_in(FM  ,pIace(amax,Z.7' ))  ]  => 
distance(Pf  .P.W  )  <  100-foot. 

Proof:  By  the  theorem  of  perception  over  time,  any  chronicle  containing  a  knowledge 
accessible  situation  must  have  a  behavior  compatible  with  the  agent's  perceptions  up  to  that 
time.  But  Fred  perceives  throughout  the  time  interval  iO  that  some  point  of  Max  is  within  ten 
feet  of  him,  so  any  knowledge  accessible  chronicle  must  have  some  point  of  .Max  within  ten 
feet  of  Fred  at  all  times.  .Moreover.  Fred  knows  that  Max  has  a  diameter  of  less  than  seven 
feet,  so  that,  in  any  knowledge  accessible  chronicle,  any  point  of  Max  is  less  than  seven  feet 
from  any  other  point  of  Max.  By  the  triangle  inequality,  then,  in  any  knowledge  accessible 
chronicle  during  lO.  any  point  of  .Max  is  less  than  seventeen  feet  from  Fred. 

V.  Judy,  Sharon,  and  the  bus  are  convex  objects.  .No  other  object  intersects  the  convex 
hull  of  the  three  of  them.  We  assume  that  Sharon  has  a  unique  set  of  visible  properties  that 
allow  her  to  be  distinguished  from  anyone  else.  Both  Judy  and  Sharon  recognize  busses;  that 
is  busses  have  visual  properties  that  Judy  and  Sharon  know  attach  only  to  them.  Due  to  the 
geometry  of  the  situation,  Judy  can  see  Sharon,  the  bus,  and  the  space  berween  them.  We 
wish  to  show  that  Judy  knows  that  Sharon  knows  that  there  is  a  bus  in  front  of  her. 

The  formal  statement  of  the  conclusion  is  as  follows:  Let  50  be  the  real  situation,  let  51 
be  any  situation  accessible  from  50  through  Judy's  knowledge,  and  let  52  be  any  situation 
accessible  from  5  1  through  Sharon's  knowledge.  Then  in  5  2  there  is  a  bus  in  front  of  Sharon. 

[  k(ajudy.50,51)  /  k(asharon,5  1,52)  ]  =■ 
[3 OBIS  true_in(52,bus(C»fiL'5))  ' 

in_front{place(Ofli,'5  ,layout(521),  place(asharon,layout{52)))  ] 

The  statement  that  Judy  recognizes  busses  can  be  formalized  as  the  statement,  "In  any 
situation  knowledge  accessible  to  Judy,  any  object  with  the  same  visual  properties  as  some 
bus  is  also  a  bus.  " 

[  k(ajudy, 50,51)  '  same_vprops(0  ,0flL'5 )  a  true_in(5  1  ,bus(0flL'5  ))  ]  ^■ 
true_ini5  1,busfO)) 

Proof:  Since  Judv's  view  of  Sharon,  the  bus.  and  the  space  between  them  is  unblocked, 
any  layout  visually  compatible  with  her  perceptions  will  have  to  have  Sharon  in  Sharon's 
place,  some  object  indistinguishable  from  a  bus  in  the  bus's  place,  and  the  space  between 
unblocked.  Since  Judy  recognizes  busses,  the  object  indistinguishable  from  a  bus  must, 
indeed,  be  a  bus.  By  A. 7,  that  knowledge  accessible  states  have  visually  compatible  layouts, 
this  same  geometrical  constraint  must  apply  in  any  situation  51  knowledge  accessible  to  Judy. 
Thus,  Sharon's  view  of  the  bus  in  the  layout  of  5  1  is  unblocked:  therefore,  in  any  layout 
visually  compatible  with  5  1  to  Sharon,  there  must  be  a  bus  in  its  place.  Therefore,  applying 
axiom  A. 7  again,  in  5  1  Sharon  must  know  that  there  is  a  bus  in  front  of  her. 

5.    The  Problem  with  Inferring  Ignorance 

Our  proof  of  inference  (I),  that  Claire  cannot  know  whether  the  mystery  object  is  mov- 
ing or  still,  rests  on  axiom  A. 8,  which  limits  what  an  agent  can  know  about  the  course  of 
events  to  what  he  can  perceive.  Specifically,  the  axiom  states  that,  if  a  course  of  events  is 
possible  (we  have  above  specified  "physically  possible",  but  broader  classes  of  possibility  can 
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be  considered)  and  it  is  compatible  with  the  agent's  perceptions  over  time,  then  the  aeent 
cannot  know  that  this  is  not  the  course  of  events  that  took  place.  Clearly,  some  such  axiom  is 
needed  if  we  are  to  deduce  the  ignorance  of  agents  from  the  limits  of  their  perceptions. 

L'nfortunatelv.  axiom  A. 8  is  so  strong  that  it  rules  out  many  plausible  states  of 
knowledge.  This  problem  is  particularly  exacerbated  by  the  weakness  of  our  physics,  which 
allows  all  kinds  of  courses  of  events  as  possible.  For  example,  when  I  first  formulated  exam- 
ple (III).  I  wanted  to  specify  that  Joanne  knew  that  she,  the  cat,  and  the  room  were  the  only 
objects  in  the  enclosure.  However,  axiom  A. 8  makes  this  impossible.  Whatever  Joanne  has 
done  or  seen  in  the  past,  her  perceptions  were  necessarily  consistent  with  the  course  of  events 
in  which  there  is  a  swarm  of  small  mobile  objects  that  have  until  now  staved  hidden  from  her 
on  the  opposite  sides  of  the  room.  Therefore  by  axiom  A. 8,  she  does  not  know  that  this 
swarm  does  not  exist.  If  so,  then  there  is  no  way  that  she  can  predict  that  she  will  ever  see 
either  the  cat  or  the  room  again,  since  the  swarm  could  at  any  moment  come  and  surround 
her  and  block  her  entire  vision. 

Another  example:  It  would  be  nice  to  augment  the  physics  by  allowing  certain  objects  to 
be  specified  as  immobile.  Such  a  physics  would  be  substantially  more  realistic,  and  would 
simplify  many  inferences.  For  example,  as  things  stand  now.  in  example  III,  Joanne  must 
know  that  she  will  see  enough  of  the  inside  of  the  room  to  know  that  she  is  inside;  this  would 
not  be  necessary  if  Joanne  could  know  that  the  room  is  fixed.  It  is,  in  fact,  easy  enough  to 
change  the  physical  axioms  to  accommodate  this;  all  that  is  required  is  to  add  the  requirement 
:hat  immobile  ob-ects  do  not  change  position  m  any  physically  possible  behavior.  However, 
under  Axiom  A.b,  it  is  necessary  either  that  everyone  always  know  which  objects  are  immo- 
bile, or  that  no  one  ever  knows  which  objects  are  immobile.  If  we  define  immobility  as  a 
situation-independent  property  of  objects,  then  each  agent  must  always  know  the  mobility  of 
each  object.  If  we  al'ow  an  object  to  be  immobile  in  one  behavior  and  mobile  in  another, 
then  there  is  no  way  to  distinguish  visually  between  the  case  where  the  object  is  actually 
immobile,  and  can  be  trusted  to  stay  in  the  same  place,  and  the  case  where  it  is  mobile  in 
principle  but  has  simply  chosen  to  stay  in  the  same  place  up  till  now.  Since  thev  are  not  visu- 
allv  distinguishable,  an  agent  cannot  know  which  is  the  case.  Of  course,  we  could  eliminate 
this  particular  problem  by  properly  enriching  the  physics  with  a  rule  such  as  "If  a  mobile 
object  is  hit,  it  will  necessarily  move."  but  how  are  we  to  know  that  the  new  physics  will  not 
suffer  from  the  same  problem  as  regards  some  other  property ■;•  In  general,  given  a  physical 
theorv  and  a  property  in  that  theory,  it  is  difficult  to  tell  whether  it  is  consistent  with  axiom 
A. 8  that  agents  should  sometimes  but  not  always  know  the  value  of  that  property.  More- 
over, It  is  objectionable  to  have  our  epistemic  theory  dictate  the  form  of  our  physical  theory. 

Thus,  axiom  A. 8  in  its  current  form  rules  out  problem  formulations  that  seem  entirely 
natural.  .Moreover,  it  does  so  in  quite  subtle  wavs.  It  took  me  personally  a  while  to  discover 
the  inconsistency  in  the  original  formulation  of  example  III;  it  would  probably  not  be  feasible 
to  do  so  automatically.  Even  worse,  it  becomes  quite  difficult  to  convince  oneself  that  a  prob- 
lem formulation  is  consistent.  Each  time  you  specify  that  an  agent  knows  something,  you 
have  to  ask  yourself  whether  he  could  possibly  know  it,  given  axiom  A. 8. 

For  example,  we  specify  in  example  IV  that  Fred  knows  that  .Max  has  a  diameter  of  less 
than  seven  feet.  Before  going  further  with  the  problem,  we  have  to  check  whether  this  is  pos- 
sible: can  Fred  possibly  know  this?  As  it  happens,  the  answer  is  that  be  can.  under  the  proper 
circumstances.  Two  things  are  required:  Fred  must  know  that  this  object  he  is  seeing  has 
diameter  less  than  seven  feet,  and  he  must  know  that  it  is  .Max.  Fred  can  know  the  diameter 
if  be  has  walked  around  .Max  and  seen  all  sides  of  him.  .Moreover,  he  must  know  that  be  had 
seen  ail  sides  of  .Max.  This  is  a  non-trivial  requirement.  For  example,  as  far  as  I  can  tell, 
there  is  no  way  that  anyone  in  this  theory  can  ever  know  that  an  object  is  circular.  Even  if 
you  walk  around  it,  you  don't  know  whether  you  have  seen  all  sides  of  it,  or  whether  it  has 
been  rotating  with  you,  like  the  moon,  and  keeping  the  irregularities  on  its  backside  hidden 
from  you.  For  Fred  to  know  that  the  object  he  sees  is  .Max,  we  must  posit  that  .Max  is  visu- 
ally distinguished  from  every  other  object;  if  Max  had  a  Doppelganger,  then  there  is  no  way 
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for  Fred  to  know  that  he  is  not  seeing  the  Doppelganger.  As  this  analysis  suggests,  in  this 
theory  justitying  a  problem  statement  by  constructing  scenarios  in  which  the  agents  have  the 
stated  knowledge  can  be  as  much  work  as  solving  the  problem. 

The  system  does  allow  great  freedom  in  specifying  knowledge  about  situation  dependent 
properties;  if  a  property  or  relation  is  not  an  element  of  the  layout,  axiom  A. 8  does  not 
apply,  and  hence  does  not  rule  knowing  about  it.  Thus,  we  tend  to  define  as  much  as  we  can 
as  situation-dependent  properties;  in  the  above  examples,  we  have  treated  "being  a  cow"  and 
"being  a  bus"  as  situation  dependent.  Axiom  A. 8  does  not  prevent  us  from  knowing  that 
cows  are  larger  than  breadboxes,  because  that  fact  does  not  constrain  layouts  in  any  way.  It  is 
consistent  with  any  layout;  it  affects  only  how  the  situation  categorizes  the  objects  of  the  lay- 
outs as  cows  or  non-cows.  This  freedom,  however,  suffers  from  two  critical  limitations.  The 
first  reason  is  that  one  can.  so  to  speak,  tie  down  a  situation  dependent  property  to  a  physical 
lavout  only  at  one  point;  if  you  try  to  tie  it  down  at  two  points,  you  run  into  trouble.  For 
example,  an  agent  can  know  that  a  cow  is  larger  than  a  breadbox;  alternatively  an  agent  can 
be  able  to  recognize  a  cow;  that  is.  to  know  that  anything  of  specified  visible  properties  is  a 
cow.  However,  an  agent  cannot  do  both.  If  he  did,  he  would  know  that  anything  with  those 
properties  is  larger  than  a  breadbox.  This  is  not  allowed  by  axiom  A. 8,  since  there  can  per- 
fecth'  well  be  layouts  in  which  there  are  objects  smaller  than  breadboxes  with  the  given  pro- 
perties. 

The  second  limitation,  which  is  related  to  the  first,  is  that  what  we  are  allowing  agents 
to  know  IS  actually  trivial,  particular  in  the  context  of  our  theory,  which  so  strongly 
emphasizes  knowledge  of  physical  facts.  Our  theory  allows  an  agent  to  know  that  there  are 
no  unicorns,  if  "unicorn"  is  taken  as  a  situation  dependent  property.  However,  it  does  not 
allow  the  agent  to  know  that  there  are  no  horse-shaped  objects  with  a  single  horn,  which  is 
wnat  he  really  wants  to  know  when  push  comes  to  shove.  We  can  fix  this  by  containing  lay- 
outs including  such  objects  as  impossible.  However,  this  has  the  unfortunate  consequence 
that  now  all  agents  must  know  that  there  are  no  unicorns;  it  is  no  longer  possible  for  one 
agent  to  know  it  and  for  another  agent  to  be  ignorant  ot  it. 

One  possible  fix  is  to  change  A. 8  to  read  as  follows:  "If  a  behavior  B  is  compatible  with 
an  agents  perceptions,  and  if  the  agent  does  not  know  that  B  is  impossible,  then  the  agent  can- 
not know  that  B  has  not  occurred."  This  formulation  is  clearly  m  some  ways  more  reasonable 
than  the  formulation  that  we  have  put  forward,  since  it  allows  agents  to  have  different 
degrees  of  knowledge  about  what  is  possible.  If  one  agent  knows  more  physics  than  another, 
he  will  be  able  to  make  tighter  predictions  about  what  will  happen.  However,  from  the  point 
of  view  of  developing  a  useful  theory  for  inference,  this  axiom  throws  out  the  babv  with  the 
bathwater.  The  point  of  the  entire  exercise  was  to  be  able  to  deduce  that  the  agent  was 
ignorant  of  one  fact,  purely  by  knowing  about  what  be  has  seen,  without  knowing  that  he  is 
ignorant  of  some  other  fact.  By  contrast,  to  use  this  axiom  to  deduce  that  the  agent  does  not 
know  that  B  does  not  occur,  we  must  be  first  be  able  to  deduce  that  the  agent  does  not  know 
that  B  is  not  impossible.  The  latter  deduction  involves  the  same  kinds  of  problems  as  the 
first.  The  only  case  where  it  is  easy  to  deduce  that  the  agent  does  not  know  that  B  is  impossi- 
ble is  when  B  is,  in  fact,  possible;  but  this  is  just  the  case  covered  by  our  original  formula- 
tion. 

Another  possible  fix  would  be  to  change  the  axiom  to  read,  "If  behavior  B  is  compati- 
ble with  the  agent's  perceptions  between  times  Tl  and  72.  and  the  agent  does  not  know  in 
n  that  B  does  not  occur,  then  he  cannot  know  in  T2  that  fl  does  not  occur."  In  other 
words,  perception  limits  what  an  agent  can  find  out  but  not  what  he  can  know.  (Other 
knowledge  would  be  innate.)  Unfortunately,  this  axiom  turns  out  to  be  too  restricted  to  be 
applied  in  most  cases.  Consider  example  I,  where  we  are  trying  to  deduce  that  Claire  does 
not  know  whether  the  object  is  moving  or  is  standing  still.  The  inference  we  would  want  to 
make  is  as  follows:  If  Claire  does  not  know  at  the  beginning  of  iO  whether  the  object  will 
move  or  not.  then,  since  what  Claire  can  see  is  physically  consistent  either  with  the  object 
moving  or  with  its  standing  still.  Claire  will  not  know  at  the  end  of  iO  whether  the  object  has 
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moved  or  not.  Unfortunately,  this  does  not  follow,  since  Claire  might  know  any  number  of 
things  that  relate  what  she  can  see  to  what  the  object  does.  For  instance,  it  is  consistent  both 
with  the  axiom  and  with  the  problem  formulation  that  Claire  should  know  the  statement, 
"The  object  will  move  during  iC  if  and  only  if  the  cat  (assumed  to  be  visible)  moves  during 
iO."  If  we  allow  Claire  to  know  lacts  like  this  innately,  then  the  inference  is  not  legitimate;  it 
IS  hard  to  disallow  facts  like  this  in  a  general  rule,  and  clumsy  to  do  so  in  the  particular  prob- 
lem formulation. 

Our  problem  here  is  to  distinguish  between  facts  like,  "There  is  nothing  that  looks  like  a 
horse  with  a  horn,"  which  we  want  to  allow,  but  not  compel,  agents  to  know,  and  facts  like, 
"The  object  will  move  during  iO  if  and  only  if  the  cat  moves  during  iO,"  which  we  wish  to 
compel  agents  not  to  know.  Intuitively,  the  difference  is  that  the  first  statement  is  a  general 
law,  which  can  be  inferred  inductively  from  the  failure  to  encounter  any  unicorns,  while  the 
second  is  neither  a  general  law  nor  an  instance  of  any  general  law.  However,  it  does  not 
appear  that  a  possible  worlds  theory  supports  this  kind  of  distinction;  we  may  have  to  move 
to  a  syntactic  theory  of  knowledge  to  accommodate  it. 

The  ultimate  problem  here  is  that  the  only  kind  of  knowledge  acquisition  we  are  consid- 
ering is  deductive  inference  through  perception,  and  this  is  simply  not  adequate  for  many 
t\'pes  of  knowledge.  V*'e  have,  in  fact,  fallen  into  Berkeley  s  position,  that  nothing  can  be 
known  except  the  pure  perceptions,  except  that  we  have  arbitrarily  given  agents  innate 
knowledge  of  what  is  physically  possible  and  impossible.  In  reality,  if  a  person  explores  a 
territorv  and  sees  nothing,  he  is  entitled  to  believe  that  there  is  nothing  there,  and  to  discount 
the  possibilitv  that  something  is  cunningly  hiding  from  him.  If  he  is  right,  we  can  call  this 
knowledge.  Our  theory  of  knowledge,  like  most  such  theories,  approximates  rationality  in 
terms  of  the  axiom  of  consequential  closure,  the  assumption  that  an  agent  can  make  all  logi- 
cal deductions.  It  has  often  been  pointed  out  that  this  axiom  is  too  strong  (e.g.  [Konolige, 
86).  [Levesquc,  84)).  it  is  much  less  often  noted  that  it  is  also  too  weak,  and  that  a  complete 
account  of  knowledge  must  contain  a  description  of  plausible  reasoning. 

Another  discomfitting  feature  of  our  theory  is  more  fundamental  than  the  particular 
axiom  A. 8.  Our  theory  provides  three  ways  to  define  properties  of  objects;  they  can  be 
siruation-dependent,  layout-dependent,  or  absolute.  The  criteria  for  choosing  one  or  another 
are  not  always  intuitive  and  the  consequences  of  one  choice  or  another  are  not  always  obvi- 
ous. It  often  requires  a  delicate  balancing  act  to  come  up  with  a  structure  that  is  powerful 
enough  to  support  desired  conclusions,  but  not  so  powerful  as  to  support  preposterous  con- 
clusions. Even  if  a  property  is  conceived  as  a  necessary  property  of  an  object,  it  may  make  a 
difference  whether  it  is  axiomatized  as  holding  in  all  situations,  holding  in  all  layouts,  or  sim- 
ply holding  absolutely. 

6.    Previous  Work 

Little  work  has  been  done  in  AI  on  reasoning  about  perception.  ATTEND,  the  focuss- 
ing of  a  sensory  organ,  was  a  primitive  act  in  conceptual  dependency  [Schank.  75)  and  was 
causally  connected  to  MBUILD.  the  performance  of  a  mental  act:  but  the  logic  of  these  acts 
was  not  developed  in  detail.  A  number  of  theories,  such  as  [Appelt,  82),  [Allen  and  Perrault. 
80)  and  [.Morgenstern,  87),  have  studied  the  acquisition  of  knowledge  through  communica- 
tion, but  these  have  not  looked  at  direct  sensation.  The  definition  of  a  perception  as  a  set  of 
physical  layouts  was  put  forward  in  [Davis,  84).  Hintikka  (1969)  used  a  modal  theory  of  per- 
ceptions to  eliminate  the  need  for  sense-data  as  ontological  primitives.  The  situation  seman- 
tics of  [Barwise  and  Perry,  82)  studied  the  logical  structure  of  sensory  verbs  in  detail,  but  did 
not  relate  it  to  knowledge  acquisition  or  to  physical  constraints.  Reiter  and  .Mackworth  (1987) 
give  a  formal  account  of  the  relation  between  an  image  and  a  physical  situation. 
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7.    Conclusions  and  Future  Work 

We  have  given  a  formalism  in  which  a  few  basic  problems  relating  perception  and 
knowledge  can  be  stated  and  solved.  A  few  features  of  our  approach  should  be  pointed  out: 

There  are  two  significant  technical  innovations  in  this  work.  The  first  is  the  introduction 
of  the  concept  of  a  physical  layout,  which  specifies  just  the  physical  state  of  the  world,  and 
the  description  of  perception  in  terms  of  layouts.  The  second  is  the  axiom  A. 8,  which  limits 
an  agent  s  knowledge  of  contingent  physical  facts  of  certain  types  to  that  which  can  be 
deduced  from  his  perceptions,  together  with  physical  laws.  This  axiom,  however,  places  con- 
straints on  an  agents  knowledge  that  are  unacceptably  strong  in  many  cases.  In  most  practi- 
cal problems,  the  positive  inference.  "Since  A  sees  <b,  A  knows  cb,"  is  more  important  that 
the  negative  inference,  "Since  A  doesn't  see  cb,  A  cannot  know  <i>:"  hence,  it  may  be  best  to 
drop  axiom  A. 8  or  to  restrict  its  scope. 

The  theory  of  the  connection  berween  perception  and  knowledge  is  largely  independent 
of  the  physical  theory.  In  particular,  axioms  A.l  through  A. 7  may  be  used  together  with  any 
set  of  physical  laws,  and  with  any  laws  delimiting  the  powers  of  vision  in  terms  of 
bv_compatibility.  Axiom  A.H  cam  also  be  used  with  any  definition  of  physical  laws  and  laws 
of  vision,  but  it  may  put  strong  limits  on  the  range  of  agents'  knowledge  of  physical  proper- 
ties; that  is.  It  may  require  that  either  all  agents  always  know  the  value  of  a  particular  physi- 
cal property  or  that  no  agent  ever  know  it. 

We  used  a  possible-worlds  theory  of  knowledge,  as  in  [Moore,  80]  rather  than  a  syntac- 
tic theory  of  knowledge,  as  in  (Konolige,  86j  for  a  couple  of  reasons.  First,  there  is  a  techni- 
cal difficulty  with  syntactic  theories.  In  a  possible  worlds  theory,  we  can  easily  say  that  a  per- 
son knows  facts  about  each  connected  subset  of  the  plane.  However,  this  is  an  uncountable 
set;  hence,  a  syntactic  theory  must  either  be  restricted  to  sets  definable  in  first-order  formu- 
las, or  must  start  with  an  uncountable  alphabet  of  symbols.  Second,  the  use  of  physically  pos- 
sible layouts  and  behaviors  is  basic  to  this  theory.  These  concepts  go  well  with  possible 
worlds  semantics;  their  use  in  a  syntactic  theory  seems  forced.  Finally,  we  simply  had  no 
need  for  the  power  of  a  syntactic  theor\'.  However,  we  plan  to  study  reformulations  of  this 
theory  in  which  "know "  is  a  syntactic  operator. 

We  plan  to  extend  this  theory  in  several  ways.  We  hope  to  incorporate  more  realistic 
theories  of  vision  and  of  phvsics  and  to  add  other  modes  of  perception;  to  study  the  connec- 
tion between  perception  and  prior  knowledge  in  greater  depth,  in  particular  the  issues 
involved  in  recognizing  specific  objects;  to  integrate  our  theory  with  a  theory  of  communica- 
tion through  writing  and  speech  and  with  a  theory  of  plans  [.Morgenstern.  87];  and  to  apply 
our  theory  to  the  problem  of  forming  cognitive  maps  [Davis,  86a]. 

8.    Appendix:  Formal  Proof 

In  this  appendix,  we  give  a  formal  proof  of  our  first  sample  inference.  We  use  a  first- 
order  sorted  logic  with  equality.  Since  our  primary  interest  here  is  in  perception  and 
knowledge,  rather  than  physics  and  geometry,  we  have  taken  many  fairly  high-level  geometr- 
ical and  physical  theorems  as  axioms.  It  would  be  possible  to  derive  these  from  basic  axioms 
in  well-known  ways;  however,  the  resultant  proofs  would  be  unacceptably  long,  and  largely 
irrelevant  to  our  purposes.  We  have  explicitly  avoided  the  use  of  set  theoretic  notation,  in 
order  to  illustrate  that  problems  of  this  kind  require  only  very  shallow  sets,  and  in  order  to 
simplify  the  structure  of  sorts  needed.  If  set  theory  were  added,  it  would  be  possible  to 
replace  some  of  the  non-logical  symbols  by  standard  set-theoretic  relations,  and  to  eliminate 
some  of  the  axioms. 
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The  sorts  of  variables  and  constants  in  our  loEic  will  be  indicated  by  their  first  letter, 
using  the  key  in  table  1: 

First  Letter        Sort 


A 

Agent 

B 

Behavior 

C 

Chronicle 

I 

Closed  Interval  of  Time 

L 

Layout 

0 

Object 

P 

Geometric  Point 

S 

Situation 

T 

Instant  of  Time 

X 

Point  set 

Table  1:  Logical  Sorts 

"Agent"  is  a  sub-sort  of  "object";  hence,  any  position  that  can  accept  an  argument  of 
sort  "obiect"  can  accept  an  argument  of  sort  "agent",  though  not  vice  versa. 

We  will  indicate  variables  by  symbols  beginning  with  upper-case  letters:  symbols  begin- 
ning with  lowercase  letters  will  be  non-logical  (predicate,  function,  or  constant)  symbols. 
Free  variables  are  taken  to  be  universally  quantified  with  a  scope  of  the  entire  formula. 

Definitions  are  sometimes  followed  by  logical  formulas.  Sometimes  these  are  equivalent 
in  meaning;  sometimes,  they  capture  only  part  of  the  meaning.  Whatever  is  in  a  definition 
that  is  not  expressed  in  a  following  logical  formula  is  given  only  as  an  informal  explanation. 
The  proots  rest  only  on  the  logical  formulas  given  for  the  definitions  and  axioms. 

8.1.    Geometry 

Definition  1.1:  The  predicate  "point_in;/'  .X )"  holds  HP  is  a  point  in  X . 
Axiom  1.2:  A  point  set  is  determined  by  the  points  in  it. 
.Y1  =  A:2  -  Vp  point_in(P  ,A'l)  =  point_in(P  ,A'2) 

Definition  1.3:  The  predicate  "sub.placefX  1,X  2) "  if  .Y  1  is  a  subset  of  A"  2. 
sub_place(A:i„Y2)  -  Vp  point_in(P  .A  1)  =  point.infP  .X2) 

Lemma  1.4:  The  sub_place  relation  is  a  partial  ordering. 

sub_place(Al,Xl). 

[  sub_place(Xl,X2)  /  sub_place(X2.X  1)  ]  =  X1  =  X2. 

[  sub_place(Xl,X2)  /  sub_place{X2,X3)  ]  =■  sub_place(X  LX3) 

Proof:  Immediate  from  definition  1.3  and  axiom  1.2. 

Definition  l.S:  The  predicate  "intersect(X  1,X2)"  holds  if  the  t^*o  point  sets  intersect. 
intersect(X  1,X2)  =  3  p  point_in(P  ,A'  1)  a  point_in(/'  ..V2) 

Definition  1.6:  The  function  "intenor{X  )"  maps  a  point  set  X  to  its  interior.  (Interior 
here  is  used  in  the  topological  sense;  the  interior  of  X  is  X  minus  its  boundary.) 

.\xiom  1.7:  If  X  1  is  a  subset  of  X2.  then  the  interior  of  X  1  is  a  subset  of  the  interior  of 
X2. 

sub_place(X  1,X2)  =■  sub_place(interior(X  l).interior(X  2)) 
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Definition  1.8:  The  predicate  "overiap(A' 1,A'2)"  holds  the  interiors  of  A' 1  and  X2  inter- 
sect. 

overlap(X  1,X2)   =:  intersect(interior(A' l).interior(X2)) 

Lemma  1.9:  If  X  1  overlaps  XA   and  XA   is  a  subset  of  XB  ,  then  A'  1  overlaps  XB  . 
[  overlapt.Y  l.XA  )   ■  sub_place(A:,4  .XB  )  ]  =  overlap(X  1,XB  ) 

Proof:  Immediate  from  Definitions  1.3.  1.5.  and  1.8  and  axiom  1.7. 

Definition  1.10:  The  predicate  "connected(A  )"  holds  if  X  is  a  connected  set  of  points. 

Definition  1.11:  The  predicate  "strici_inside(X  l.X  2)"  means  that  X  1  is  a  subset  of  X  2 
and,  moreover,  their  boundaries  are  disjoint. 

strict_inside(A'  1..Y2)  ^  sub_place(A  l.X  2) 

Definition  1.12:  The  function  "line_seg(P  1  .P  2)"  maps  two  points  PI  and  P2  onto  the 
line  segment  connecting  them  (a  point  set). 

.4xiom  1.13:  "Line_seg(/'  l,P2)"  is  a  symmetric  function. 
\ine_scg[Pl.P2)  =   line_seg(P  2.P  1) 

Axiom  1.14:  PI  lies  on  the  line  segment  from  PI  to  P  2. 
point_in(P  l,line_seg(P  1,P2)) 

Definition  1.15:  The  predicate  "blocked(P/^  .X  .PB  )"  means  that  point  set  A'  blocks  the 
view  of  point  PA  from  point  PB  .  (The  order  corresponds  to  the  physical  layout:  X  comes 
between  P/1   and  Pfi  .).  That  is.  the  line  from  P.4   to  Pfi   intersects  the  interior  of  X  . 

blocked(P/i  ,A'  .PB  )  -  mtersect(interior(A: ),  line_seg(P/t  ,Pfl  )) 

We  will  overload  the  predicate  "blocked"  to  take  point-set  arguments  either  in  its  second 
and  third  arguments  or  m  all  three  arguments. 

Definition  1.16:  The  predicate  "blocked! P  ,.X'B  ..VC  )"  means  that  P  is  blocked  by  XB 
every  point  in  XC  .  The  predicate  "blocked(.X'y4  ,XB  ,XC  )"  means  that  each  point  of  XA  is 
blocked  by  XB  from  any  point  m  XC  . 

blocked(P.A'B  .XC  )   =  V  p^-  [  point_in(PC,XC)  =  blocked(P  ,X5  .PC)  ] 
blockedfX.4  .XB  .XC  )   -  V  r.   [point_in(PA  ,X,4  )  =   blocked(P/l  .XB  .XC  )  ] 

Lemma  1.17:  If  XA  is  blocked  by  XB  from  XC  .  then  any  subplace  of  XA  is  blocked 
from  any  subplace  of  XC  . 

[  blocked(XA  .XB  .XC)  a  subplacefXA  1,XA  )  a  sub_place(XCl,XC)  ]  =• 
blockedlXA  l.XS,XCl) 

Definition  1.18:  The  predicate  "unblockedfXA  ,XB  .XC )"  holds  if  no  point  in  XA  is 
blocked  by  XB  from  XC  . 

unblockedfXA  .XB  .XC  )  ■= 

Vp^  [  point_in(PA  .XA  )  =■  -blockedfPA  .XB  .XC  )  ] 

8.2.    Time  and  Motion 

Definition  2.1:  The  predicate  "7"1<72"  (written  infix)  holds  if  time  instant  71  is  earlier 
or  CQual  to  T2. 
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Axiom  2.2:  Time  instants  are  totallv  ordered. 

T\^T2  .  r:<ri 

[ri<r2  A  T2^T\]  =■  Tl  =  T2 
[Tl^T2   ■  72<r3i  =  71^73 

Definition    2.3:   The    function    "startf/)"    maps   a   time    interval    /    to    its    starting     time 
instant. 

Definition  2.4:  The  function  "endf/)"  maps  a  time  interval  /  to  its  ending  time  instant. 

Definition  2.S:  The  predicate  "time_in(7 ,/ )"  holds  if  the  time  instant  7   is  part  of  the 
time  interval  / . 

time_in(7,/)  -  start(/)  s  7  <  end(/) 

Axiom  2.6:  Anv  two  ordered  unequal  time  points  determine  an  interval. 
[  71^72   •   7l7t  72]  =  3,  [71  =  start(/)  /   72  =  end(/)  | 

Definition   2.7:   The    predicate    "subintervaU/ 1,/2)"   holds   if  interval   / 1    is   a   subset   of 
interval  12. 

subinten-al(/l,/2)   =  [  V;.  timc_in(7  ,/ 1)  =  time_in(7  ./2  )  ] 

Definition  2.8:  The  function   "scenelS  .7 )"  maps  a  behavior  B    and  a  time   7   onto  the 
layout  of  B  at  time  7  . 

.Axiom  2.9:  1  wo  lavouts  are  equal  just  if  they  have  the  same  objects,  and  they  assign 
them  to  the  same  places.  (We  do  not  consider  rotations  in  place  to  make  a  difference). 

L1  =  Z.2  = 

Vo  [  obiect_of(0.7  1)   -  ob]ect_of(0  ,Z.  2  )  |  a 

;  object_of(0,Z.l)  =■  place{O.Z.l)  =  place(C>  ,L  2)  ) 

Axiom  2.10:  Two  behaviors  are  equal  just  if  corresponding  layouts  are  equal. 
Sl-fi2  =  Vt  lavout(fll.7)   =   layout(S2.7) 

Definition  2.11:  The  function  "chronicle(5  )"  maps  a  situation  S  onto  the  chronicle  con- 
taming  It. 

Definition  2.12:  The  function  "timef^)"  maps  a  situation  S  onto  the  time  when  it  occurs. 

Definition  2.13:  The  function  "s!tuation(C  .7 )"  maps  a  chronicle  C  and  a  time  instant  7 
into  the  situation  of  C  at  time  7. 

situation(chroniclel5  ),  time(5))  =  5. 

Definition  2.14:  Situation  51  precedes  S2  if  they  belong  to  the  same  chronicle  and  51 
occurs  earlier. 

precedes(5  1,5  2)  -  chronicle(51)  =  chronicle(5  2)  a  time(51)  s  time(5  2) 

Definition  2.15:  The  function  "layout(5)"  maps  a  situation  5    into  the  physical  layout 
present  in  5 . 

Definition  2.16:  The  function  "behavior(C )"  maps  a  chronicle  C  onto  its  behavior. 

.Axiom  2.17:  A  chronicle  at  a  given  time  has  the  same  layout  whether  you  go  through 
the  behavior  or  through  the  situation. 

scene! behaviorfC  ).  7)  =  layout(situation(C  ,7)) 
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Note:  The  next  several  axioms  could  all  be  proved  in  greater  generality  from  fundamen- 
tal defmition  of  a  layouts  as  a  functions  from  a  set  of  objects  to  space  and  of  a  behavior  as  a 
continuous  function  from  time  to  layouts.  To  do  so,  however,  would  require  reasoning  about 
sets  of  objects,  function  spaces,  and  continuity.  We  have  therefore  short-circuited  the  process 
by  postulating  the  results  of  direct  interest. 

.Axiom  2.18:  Two  behaviors  fil  and  fi2  which  have  equal  layouts  at  some  time  TS  may 
be  spliced  together  across  time  into  a  new  behavior  that  agrees  with  fl  1  up  to  TS  and  with  5  2 
after  TS . 

[  \a\ouUBlJS)  =   lavout(B  2,7-5)  ]  =• 
3s3  [  ^T^TS  layout(fi3,7)  =  layout(fl  l.T")  A 
Vt,t5  layout(fl3,r)  =  layout(fl2,7)  ] 

.4xiom  2.19:  Anv  two  behaviors  fll  and  fl2  can  be  spliced  together  across  objects  to 
form  a  new  behavior  B  }  which  agrees  with  B  1  on  the  objects  of  B  1  and  agrees  with  fl  2  on 
the  objects  that  are  in  fl  2  but  not  in  fl  1.  Note  that  fl  3  may  not  be  physically  possible. 

[  object_of(0,fl  1)  =■ 

[  ob]ect_of(0,B3)    ■  placefO  .layoutffl  3.7  ))  =   place(0  ,layout(B  1,7)  |]   ' 
[  [  ob]ect_of(O.B2)  :   -ob]ect_of(0  ,fl  1 )  ]  = 
[  ob]ect_of(0,fl3)   ■  place(0,layout(fl3,7))  =  place(0  .layout(fl  1.7)  |  ] 

.\xiom  2.20:  Given  a  la\out  L  and  an  object  O  in  Z.  .  there  exists  a  layout  LO  which  has 
O  in  the  same  place  as  in  Z. .  and  which  has  no  other  objects. 

object_of(0,L)  ^  3^.y  [object_of{0  LLC  )  -01=0  )   '   place(0,/.0  )  =   placet  O  ,L) 

Definition  2.21:  The  predicate  "motionlessi  O  .6  ./ )"  holds  if  the  place  occupied  by  O  in 
B  does  not  change  during  /.  (B\  this  definition,  we  are  not  counting  rotation  in  place  as 
motion.) 

motionless(0,B  ,/)  - 

Vr,  7;  [  [    time_in(71,/)  •  time_in(72,/ )  ]  =• 

place(0,scene(fl  ,7  1))  =   place(0  ,scene(fi  .72))  ] 

Definition  2.22:  The  predicate  "continual_motion(0  ,fl  ,/ )"  holds  if  O  is  never  motion- 
less in  fl  throughout / . 

continual_motion(0  ,B  ,/)  = 

[V^5  sub_intervaU/i',/ )  =   -  motionless(0  ,fl  ,/5  )  ] 

Axiom  2.23:  Given  any  layout  L  ,  there  is  a  behavior  including  L  in  which  everything  is 

motionless. 

3g  L   =  layout(fl,7f  )  '.  V  ,  ^  object_of(0  ,B  )  =•  motionless(0  .B  ./) 

Axiom  2.24:  If  an  object  0  is  strictly  inside  a  region  XE  in  a  layout  L  .  then  there  exists 
a  behavior  fl  which  includes  L  ,  in  which  0  is  in  continual  motion  but  stays  inside  XE  . 

[  ob]ect_of(0,Z.)  A  strictlyJnsideCplaceCO,/.),^^)  ]  = 
3s  [L   =  scene(fl,7f)  a  V^  strictly_inside(place(0  ,scene(fl  ,7)),X£  )  a 
V  ,  continual_motion(A'  ,fl  ,/  )  ] 

8.3.    Physics 

Definition  3.1:  The  predicate  "object_of(0  ,Z. )"  holds  if  object  O  is  within  layout  L.  The 
predicate  "object_of  (0  ,fl  )"  holds  if  object  O  is  within  behavior  B  . 
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Axiom  3.2:  A  behavior  has  the  same  objects  as  each  of  its  layouts. 
object_ol(O.S  )  ■-  object_of(0  ,scene(fi  .7)) 

Definition  3.3:  The  function  "place(0,L)"  maps  an  object  O  and  a  lavout  L  into  the 
point  set  occupied  by  O  during  L  . 

Definition  3.4:  The  predicate  ■■same_vprops(0  1,0  2) "  holds  if  01  and  0  2  have  the  same 
visual  properties. 

Axiom  3.5:  The  relation  "same_vprops(0  1,02)"  is  an  equivalence  relation. 

same_vprops(0  ,0)  / 

same_vprops(0  1,02)  =  same_vprops(02,0  1) 

[  same_vprops(01,02)  a  same_vprops(02,03)  |  =  same_vprops(0  1,03) 

Definition  3.6:  The  predicate  "free_space(A:  ,Z. )"  holds  if  point  set  .V  does  not  intersect 
the  place  of  any  bodies  in  layout  L  . 

free.spacetX  ,Z.)  ~  [^  ,-,  object_of(0  ,Z.  )  =   -intersect(A' ,place(0  ,Z.  ))  ] 

Definition  3.7:  The  predicate  "visiblefP  ,,4  .L  )"  holds  if  the  point  P  is  visible  to  agent  A 
in  lavout  L.  That  is.  there  is  no  object  between  A  and  P  .  (Note  that  each  point  in  A  itself  is 
visible  to  .4  by  definition.) 

visible{/',-4  .L)  - 

v.,  [  [object_of(O.L  )   ■  O  =?  .4   j  =■ 

3n^  point_in(/'/l  ,place{A  ,L))  /   -blocked(P  .place(0  .Z.  ),/'A  )  ] 

.\xiom  3.8:  If  a  point  P  is  invisible  to  agent  A  in  L  ,  then,  for  any  point  PA  \r\  A  ,  there 
is  some  visible  point  on  an  object  that  blocks  P  from  PA  .  (This  is  non-trivial;  it  relies  on  an 
assumption  that  there  are  only  finitely  many  objects  in  any  finite  area.  Imagine  looking  down 
at  an  infinite  stack  of  pages,  each  half  as  thin  as  the  one  underneath.  Then  none  of  the  pages 
would  be  visible.) 

[  -visible(P  .-4  ,L  )   ■  point_in(/'A  ,place(A  ,L  ))  |  = 
3o  [  blocked(/',place(0,Z.),P.4  ))   ' 

3pr,  [point_in(PO,place(0,Z.))  '  visiblefPO  ,.4  ,Z. )  /, 
point_in(PO,line_seg(PA  ,P))  )  ] 

Definition  3.9:  The  predicate  "whoily_visible(.V  ,A  ,Z.)"  holds  if  each  point  of  X  is  visible 
to  A   in  L  . 

wholly_visible(X  A  .L  )  _  [  V  p  pointJn(P  ,X  )  =  visiblefP  .A  .L)  \ 

Definition  3.10:  The  predicate  "wholly_invisible(X  ,A  .Z. )"  holds  if  no  point  of  X  is  visi- 
ble to  A  in  L  . 

wholly_invisible(.Y  ,A  .L)  ■-['i  p  point_in(P  ,X )  =•  -visible(/'  ,A  ,Z. )  | 

Definition  3.11:  The  predicate  "phys_poss(Z. )"  holds  if  L  is  a  physically  possible  layout. 
Axiom  3.12:  A  layout  is  physically  possible  if  no  two  objects  overlap. 

phys_poss(Z, )  ■!=> 

[  V^i  02  [ob]ect_of(01,Z.)  A  object_of(02,Z.)  ]  ^■ 
-overlap(place(01,Z.),place(02.Z.))  ] 

Axiom  3.13:  .\  behavior  is  physically  possible  if  each  layout  is  physically  possible. 
phys_poss(fi  )  =  V  T-  phys_poss(scene(fl  ,7)) 
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Axiom  3.14:  Anv  chronicle  is  physically  possible. 
phys_poss(behavior(C  )) 

8.4.    Perception  and  Knowledge 

Definition  4.1:  The  predicate  "v_compatible(/^  ,11.12)"  means  that  lavout  L  2  is  visually 
compatible  with  layout  LI  relative  to  agent  A  .  That  is,  as  far  as  A  can  see  in  L  1.  the  world 
might  be  in  state  L2. 

Axiom  4.2:  Lavout  Z.2  is  v_compatible  with  L  1  with  respect  to  agent  A  iff  the  following 
two  conditions  hold:  (i)  L  1  and  L  2  are  both  physically  possible,  (ii)  Let  X  be  a  connected  set 
of  points,  such  that  every  point  in  X  is  visible  to  ^4  in  L  1.  Then  each  point  of  X  is  visible  to 
A  in  Z.2.  .Moreover,  if  X  lies  entirely  in  A  in  Z.1,  then  X  lies  entirely  in  A  in  Z.2;  if  X  lies 
entirely  in  some  object  01  in  Z.  1  then,  in  Z.2,  X  lies  entirely  within  some  object  02  with  the 
same  visible  properties  as  01;  if  X  lies  entirely  in  free  space  in  LI  then  X  lies  entirely  in  free 
space  in  L  2. 

v_compatible(A  ,Z.  1.Z.2)   = 
[  object_of(/^  ,Z.l)  *.  object_of(.4  .12)   ■   phys_poss(Z.  1)       phys_poss(Z.  2)  a 
Vyo  [  connected(.V  )  and  wholly_visible(.Y  ./^  ,L  1)  )  = 
[  wholly_visible(X.A  ,Z.2)   ' 
[  sub_place(X  ,place(i4  ,L  1))   =  sub_place(X  .place(A  ,Z.  2))  )  / 
[  [  object_of(0,Z.  1)    ■  sub_place(X.place(0,L  1))  1  - 

[  3,,,  object_of(02,L2)  /  same_vprops(0  .02)  '  sub_place(X  .place(02,Z.  2))  ]  ]  A 
[  [  object_of(0,Z.2)  -  sub_piace(X  ,place(0  ,L  2))  ]  ■- 

[  3oi  object.ofiO  l.L  1)   ■  same_vprops(0  ,0  1)   •  sub_place(X  ,place(0  1,/.  1))  ]]a 
[  frec_space(.V  .1 1)  -  free_space(.X' .L2)  ]  ]  ) 

Lemma  4.3:  v_compatibility  is  an  equivalence  relation  on  layouts. 

v_compatibleM  ,Z.  ,Z. ) 

v_compatible(y4  ,Z.  1,Z.  2)  =:  v_compatible(.4  .Z.  2.Z.  1) 

[  v_compatible(A  .Z.  1,L  2)   •.  v_compatible(/^  ,Z.2.L  3)  |  ~-  v_compatible(/i  ,L  l.Z.  3) 

Proof:  Reflexivity  and  transitivity  are  immediate  from  axiom  4.2.  since  the  right-hand 
side  of  the  definition  has  throughout  the  form  "Property  of  LI  implies  same  property  of  L2." 
This  rests  on  the  fact  that  same_vprops  is  an  equivalence  relation  (axiom  3.5).  The  axiom  is 
also  nearly  symmetric  in  Z.1  and  Z.2;  the  only  missing  point  is  to  show  that,  if  Z.  1  and  Z.2 
satisfy  the  condition  of  the  axiom,  and  X  is  wholly  visible  in  Z.2  then  it  must  also  be  wholly 
visible  in  Z.1.  This  can  be  proved  by  contradiction  as  follows:  Let  L  1  and  Z.  2  be  rwo  layouts 
satisfying  the  right-hand  side  of  axiom  4.2.  and  let  X  be  a  connected  set  of  points  visible  to  A 
in  Z.2.  Suppose  that  X  were  not  wholly  visible  in  Z.1.  so  that  (by  definition  3.9)  some  point 
PX  in  X  were  invisible  in  L  1.  Let  PA  be  a  point  in  A  .  From  axiom  3.8,  we  know  that  there 
IS  in  L  1  some  object  01  with  a  visible  part  XO  that  blocks  PX  from  PA  .  By  the  right-hand 
side  of  axiom  4.2,  in  Z,2.  XO  must  lie  in  some  object  02.  But  then  O  must  block  PX  from 
PA  in  Z.2.  Since  PA  was  chosen  arbitrarily,  PX  must  be  blocked  from  every  point  in  A, 
which  means  (by  definition  3.9)  that  X  is  not  wholly  visible  in  12.  This  establishes  the  con- 
tradiction. 

Lemma  4.4:  Let  Z.1  and  Z.2  be  physically  possible  layouts  with  the  following  properties: 
(i)  The  same  objects  are  partly  visible  in  the  two  layouts,  (ii)  Any  object  which  is  partly  visi- 
ble in  the  two  layouts  is  in  the  same  position  in  both.  Then  the  two  layouts  are  visually  com- 
patible. 
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[  object_ofM  .L  1)  "  object_of(A  ,L2)  ^  phys_poss(L  1)  '  phys_possfL  2")  ' 
[  [  object.ofiCi.  1)      -wholly_invisible(0,-4  ,Z.l)  |  = 

[  ob)ect_of(0,Z.2)  /  place(O.Ll)  =  place(0.Z.2)  ]  ]  A 
[  [  object_of(0,L2)   v  -wholly_invisible(0  ,/l  ,Z.2)  ]  = 
[  object_of(0  ,i.  1)  A  place(0  ,L  1)  =  place(0  ,L2)  ]] 
]  =:  v_compatible(/t  .Z.  1X2) 

Proof:  Assume  that  the  antecedent  of  the  lemma  holds  for  some  layouts  L\  and  L2. 
Note  that,  since  A  is  always  wholly  visible  to  itself,  it  must  occupy  the  same  place  in  Z.  1  as  in 
L2.  Let  X  be  a  set  of  points  which  is  wholly  visible  in  LI.  We  will  show  (A)  that  X  is 
wholly  visible  in  Z,  2  and  (B)  that  X  is  inside  "the  same  kind  of  space"  in  Z.  2  as  in  Z.  1,  Using 
axiom  4.2,  it  will  follow  that  L  1  and  L  2  are  visually  compatible. 

.\.  The  proof  by  contradiction  is  almost  the  same  as  that  of  lemma  4.3:  Suppose  that  X 
were  not  wholly  visible  in  Z.2,  so  that  (by  definition  3.9)  some  point  PX  in  X  were  invisible 
in  L2.  Let  PA  be  a  point  in  A.  From  axiom  3.8.  we  know  that  there  is  in  L  2  some  visible 
object  O  which  blocks  PX  from  PA  .  By  the  antecedent  of  the  lemma,  O  must  occupy  the 
same  place  in  LI.  But  then  O  must  block  PX  from  PA  in  LI.  Since  PA  was  chosen  arbi- 
trarily, PX  must  be  blocked  from  every  point  in  A  ,  which  means  (by  definition  3.9)  that  X  is 
not  wholly  visible  in  Z.  1.  This  contradicts  the  hypothesis  of  the  proof. 

B.  (i)  Let  A"  be  a  subset  oi  A  in  Z.  1.  Since  ,4  occupies  the  same  space  in  Z.  1  as  in  Z,  2,  X 
must  be  a  subset  oi  A   in  Z.  2 . 

(ii)  Let  X  be  a  non-null  subset  of  O  mil.  Since  ,Y  is  wholly  visible,  O  is  not  wholly 
invisible  (Definitions  1.3,  3.9,  3.10.)  Hence,  O  occupies  the  same  space  in  Z.2,  and  X  must 
be  a  subset  of  O  in  Z,2.  Note  that,  by  axiom  i.5,  O  has  the  same  visual  properties  as  itself. 

(ill)  Let  X  be  in  free  space  in  Z  1.  Suppose  that  X  is  not  in  free  space  in  Z,2;  that  is 
(definition  3.6),  X  intersects  some  object  O  in  12.  Since  X  is  visible  in  Z,2,  O  must  be  par- 
tially visible,  so,  by  the  antecedent  of  the  lemma,  0  must  be  in  the  same  place  in  Z.  1  as  in 
Z.2.  But  then  A!  is  not  in  free  space  m  L  1.  contrary  to  hypothesis. 

Thus  all  the  conditions  of  axiom  4.2  are  satisfied,  completing  the  proof. 

Definition  4.5:  The  predicate  "bv_compatible(/^  ,B  1,B  2,7  )"  means  that  behavior  B2  is 
visually  compatible  with  behavior  fi  1  up  to  time  T.  That  is.  B  2  is  consistent  with  everything 
that  A  can  see  in  fl  1  up  to  (and  including)  time  T . 

Axiom  4.6:  Two  behaviors  are  visually  compatible  up  to  time  T  if  corresponding  layouts 
up  to  time  T  are  compatible. 

bv_compatible(A  ,B  l.B2,r5)  •= 

'i  -r^fs  v_compatible(/4  ,situation(fl  l.D.situationCS  2.7)) 

Lemma  4.7:  bv_compatibility  is  an  equivalence  relation  over  behaviors. 

bv_compatible(A  .B  ,B  .T). 

bv_compatible(A  ,fi  1,5  2,7)  =■  bv_compatible(A  ,52,fl  1,7). 
[  bv_compatible(A  ,fll,B2,7)  a  bv_compatibie(A  ,fl2,B3,7)  ]  => 
bv_compatible(A  ,fll,B3,7). 

Proof:  Immediate  from  lemma  4.3  and  axiom  4.6. 

Definition  4.8:  The  predicate  "k(A,51,52)"  holds  if  52  is  accessible  in  5 1  via  A's 
knowledge.  That  is,  52  is  consistent  with  everything  that  A  knows  in  51. 

Axiom  4.9:  Veridicality:  All  that  is  known  is  true.  Formally,  the  knowledge  accessibility 
relation  is  reflexive. 

k(A,5,5) 
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Axiom  4.10:  Positive  introspection:  li  A  knows  <i> .  then  he  knows  that  he  knows  <i>.  For- 
malh,  the  knowledge  accessibility  relation  is  transitive. 

[  k(A  ,51.52)   ■  kM  .52.53)  ]  =■  k(/\  ,51,53) 

Axiom  4.11:  .Memory:  An  agent  does  not  forget  what  he  knows.  Formally,  let  situation 
SIB  be  accessible  from  SOB  ,  and  let  SOA  precede  SOB  in  the  same  chronicle.  Then  there  is  a 
situation  5  lA  in  the  chronicle  of  5  IS  which  is  accessible  from  SOA  . 

[  k(A  ,S0fl,51S)  A  precedes(5  0A  ,5  0fl)  ]  =• 

3 5,^    [  k(-4  ,SOA  .51-4)  A  precedes(51A  ,51S)  ] 

Axiom  4.12:  Internal  clock:  An  agent  always  knows  the  time.  Formally,  if  5 1  is 
knowledge  accessible  from  50.  then  the  two  situations  occur  at  the  same  time. 

kM  ,50,5  1)  -  time(50)  =  time(5  1) 

.Axiom  4.13:  .An  agent  knows  what  he  perceives.  Formally  if  5  1  is  knowledge  accessible 
from  50,  then  the  layout  of  5  1  is  visually  compatible  with  the  layout  of  50. 

k(.4  ,50,5  1)  ^-  v_compatible(A  ,layout(50),layout(5  1)) 

.Axiom  4.14:  Perception  is  the  only  source  of  knowledge  of  the  course  of  events.  For- 
mally, let  CO  be  the  real  chronicle.  Let  S  1  be  a  behavior  that  is  visually  compatible  with  the 
behavior  of  CO  up  to  time  T  relative  to  agent  A  ;  thus,  as  far  as  A  could  have  seen  up  to  time 
7",  fi  1  could  be  the  real  behavior.  Then  it  is  consistent  with  A  's  knowledge  that  B  1  actually 
was  the  real  behavior;  that  is.  there  is  a  chronicle  CI  whose  situation  in  T  is  knowledge 
accessible  from  the  situation  of  C  0  in  7 . 

bv_compatible(/l  ,behavior(CO).fi  1.7)  =• 
3^1  [  k{A  ,situation(C0,7),situation(Cl,7))  .'SI  =  behavior(Cl)  ] 

Theorem  4.15:  Facts  perceived  over  time  are  known.  Formally,  if  5 1  is  knowledge 
accessible  from  50.  then  the  behavior  of  the  chronicle  of  50  up  50  is  visually  compatible  with 
the  behavior  of  the  chronicle  of  5  1  up  to  5  1 . 

k(A  ,50.5  1)  -■ 

bv_compatible(/l  .behavior(chronicle(50)),  behavior('chronicler5  1)).  time(50)) 

Proof:  Let  51  be  accessible  from  50.  Let  C  0  be  the  chronicle  of  50.  let  C  1  be  the 
chronicle  of  51.  and  let  75  be  the  time  of  both  50  and  51.  fBy  axiom  4.12,  they  must  have 
equal  times.)  By  axiom  4.11.  of  memory,  for  any  situation  SOA  preceding  50  there  must  be  a 
situation  51.4  preceding  51  which  is  accessible  from  SOA.  By  axiom  4.12.  that  the  agent 
knows  the  time,  these  must  have  the  same  time.  Thus,  using  definition  2.13.  we  have  the  fol- 
lowing statement:  If  7s75  then  situationfC  1 .7)  is  accessible  from  situation(C2.7).  By 
axiom  4.13,  that  perceptions  are  known,  this  means  that  the  lavouts  layout(sifuation(C0,7)) 
and  layout(situation(C  1,7))  are  visually  compatible.  Bv  axiom  2.17  these  layouts  are  respec- 
tively equal  to  scene(behavior(C0),7),  and  scene(behavior(C  1),7).  Thus,  the  two  behaviors 
behavior(CO)  and  behavior(Cl)  have  visually  compatible  layouts  at  each  time  7s 75,  so  they 
are  bv_compatible  up  to  nme  75  . 

Theorem  4.16:  Negative  introspection:  If  an  agent  does  not  know  a  fact  about  the 
course  of  events,  then  he  knows  that  he  doesn't  know  it.  Formally,  if  a  behavior  is  compati- 
ble with  an  agents  perceptions,  then,  from  any  knowledge  accessible  situation,  there  is  a 
knowledge  accessible  chronicle  exhibiting  that  behavior, 

[  bv_compatible(A  ,behaviorfCO),fi  1,7)  /.  kf A  .situationfC 0,7), 5M  )  ]  ^■ 
3^^,  [  k(-4  ,SM  ,situation(C  1,7))  ,■  bv_compatible(A  ,behavior(C  l).fl  1,7)  ] 


Proof:  Let  A  .  B  I.  CO.  SM  .  and  T  satisfy  the  antecedent  of  the  theorem.  Let  CM  be  the 
chronicle  of  SM .  By  theorem  4.15,  the  behavior  of  CM  is  visually  compatible  with  the 
behavior  of  CO.  Since,  by  lemma  4.7.  bv_compatibility  is  an  equivalence  relation,  it  follows 
that  B  1  is  compatible  with  the  behavior  of  CM  .  Therefore,  by  axiom  4.14,  A  cannot  know  in 
SM  that  B  1  has  not  occurred;  that  is,  there  exists  a  CI  with  a  situation  accessible  from  SM 
whose  behavior  is  fl  1. 

8.5.    Example  1:  Inferring  Ignorance 

The  basic  constants  of  our  example  are  the  following: 

Constants: 

aclaire  —  Claire 

cO  —  Actually  occurring  chronicle 

iO  —  Time  intersal  in  question 

omvstery  —  The  mystery  object. 

owail  —  The  wall 

xclaire  —  Place  occupied  by  Claire 

xenvelope  —  The  spatial  envelope  containing  the  object. 

xwall  —  Place  occupied  by  wall 

We  define  a  few  additional  constants  as  convenient  abbreviations: 
Definition  5.1: 


bU 

= 

behavior(cO) 

— 

The  real  behavior. 

ta 

= 

start(iO) 

— 

Starting  time 

tz 

= 

end(iO) 

— 

Starting  time 

la 

= 

scene(bO.ta) 

— 

Starting  layout 

sOz 

= 

situation(cO, 

tz) 

— 

The  ending  situation 

Hypothesis  5.2:   Claire,   the  wall,   and   the   mystery   object   are   distinct  objects   in    the 
chronicle. 

obiect_of(aclaire.bO)   v  object_of(owall.bO)  "  ob]ect_of(omystery,bO)  a 
aclaire  ^  owall  "  owall  t^  omystery  /  aclaire  *  omystery 

Hypothesis  5.3:  Claire  occupies  the  fixed  place  xclaire  throughout  the  interval  iO. 
time_in(r  ,iO)  =■  place(aclaire,  scene(b0,7" ))  =   xclaire 

Hypothesis  5.4:  The  wall  occupies  the  fixed  place  xwall  throughout  the  interval  10. 
nme_in(r,iO)  =■  place(owall,  scene(bO,  7"))  =  xwall 

Hypothesis  5.5: 

The  mystery  object  remains  strictly  inside  the  envelope  throughout  the  interval  iO. 

time_in(r  ,iO)  =■  strict_inside(place(omystery,scene(bO,r)),  xenvelope) 

Hypothesis  5.6:  No  other  object  comes  inside  the  envelope  within  the  interval  iO. 
[  OT^omystery  ^  time_in(7,iO)  ]  =>  -overlap(place(0 ,  scene(bO,r)),  xenvelope) 

Hypothesis  5.7:  The  envelope  is  blocked  by  the  wall  from  Claire. 
blocked(xenvelope,  xwall,  xclaire) 

To  prove:  Claire  does  not  know  whether  or  not  omystery  was  motionless  during  iO.  We 
express  this  as  follows:  There  is  a  situation  accessible  to  Claire  in  situation  sOz  (the  ending 
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situation)  that  follows  on  a  chronicle  in  which  omystery  was  motionless  throughout  10.  There 
IS  also  an  accessible  situation  that  follows  on  a  chronicle  in  which  it  was  not  motionless. 

[3^,  k(aclaire.sOz,situation(C  l.tz))  a  motionless(omystery.behavior(C  l),iO)  |  /. 
[3(-,  k(aclaire,s0z.siruation(C2.tz))  a  -motionless(omystery,behavior(C2),iO)  ) 

Lemma  5.9:  There  is  a  behavior,  which  we  henceforth  call  "bl".  containing  the  same 
objects  as  bO.  such  that  (i)  all  layouts  of  bl  are  the  same  as  those  of  bO.  up  to  and  including 
time  ta;  (ii)  every  object  except  omystery  has  the  same  behavior  in  bl  as  in  in  bO;  (iii)  omys- 
tery IS  motionless  after  time  ta. 

3^,  Vp  [  object  _of(O.Bl)  tz  ob]ect_of(C>  ,bO)  ]  A 
V^^,^  layout(i?l,7")  =  layout(bO,r)  A 
[  [object_of(0  ,5  1)  A  O  ^t  omystery  ]  =• 

Vj  place_of(0,scene(fl  1,7))  =  place_of(0  .scene(bO,r))  ]  A 
V,  [ta  <  start(/)  =  motionless(omystery.6  1,/ )  ] 

Proof:  By  axiom  2.20.  there  is  a  layout  LAP  whose  onlv  object  is  omystery.  placed  in 
the  same  place  as  in  the  layout  "la".  By  axiom  2.23,  there  is  a  behavior  BX  containing  LAP  at 
time  ta  with  omvstery  motionless  throughout  BX  .  By  axiom  2.19,  there  is  a  behavior  BY 
combining  the  behavior  of  omystery  from  BX  and  the  behavior  of  the  other  objects  from  bO. 
From  the  construction  of  BY .  and  from  axiom  2.9,  it  follows  that  the  layout  of  BK  at  time  ta 
IS  equal  to  the  la\out  la.  Therefore,  by  axiom  2.18,  we  can  define  bl  to  be  the  behavior 
which  agrees  with  bO  up  to  time  ta,  and  with  BY  after  ta.  The  above  properties  follow 
directly  from  the  construction. 

Lemma  5.10:  There  is  a  behavior,  which  we  henceforth  call  "b2",  containing  the  same 
objects  as  bO,  such  that  (i)  all  layouts  of  b2  are  the  same  as  those  of  bO,  up  to  and  including 
time  ta:  (ii)  every  object  except  omystery  has  the  same  behavior  in  b2  as  in  in  bO;  (iii)  after 
time  ta,  omystery  stays  inside  xenvelope  in  a  state  of  continual  motion. 

3.,  Vy  [  object_of(C>,Sl)  -  object_of(0  ,bO)  )  A 
Vrsra  layout(fil,7")  =  layouKbO,/")  A 
[  [object_of(0  ,B  1)  '  O  ^omystery  ]  =■ 

Vj  place_of(0,scene(Bl,7'))  =  place_of(0  .scene(bO,r))  ]  a 
V  T^.j  strictly_inside(pIace(omystery  ,scene(B  2, 7), xenvelope)  a 
V,  [ta  s  start(/)  =•  continual_motion(omystery,fl  1,/)  | 

Proof:  Exactly  the  same  as  that  of  lemma  5.9,  except  that,  instead  of  using  axiom  2.23 
to  construct  a  behavior  in  which  omystery  stands  still,  we  use  axiom  2.24  together  with 
hypothesis  5.5  to  construct  a  behavior  in  which  omystery  is  in  continuous  motion  within  xen- 
velope. 

Lemma  5.11:  In  behavior  bl  at  all  times  after  ta,  omystery  is  inside  xenvelope. 
^r^m  strictly_inside(place(omystery,scene(bl,r)),  xenvelope) 

Proof:  By  hypothesis  5.5,  omystery  is  inside  xenvelope  in  layout  la.  By  lemma  5.9,  the 
layout  of  bl  at  ta  is  equal  to  la.  By  lemma  5.9,  ol  is  motionless  in  bl  in  all  intervals  starting 
after  or  at  time  ta.  By  definition  2.21  and  axiom  2.6,  this  means  that  ol  always  occupies  the 
same  place,  inside  xenvelope,  at  all  times  after  ta. 

Lemma  5.12:  Behaviors  bl  and  b2  is  both  physically  possible. 
phys_poss(bl)  ^  phys_poss(b2) 

Proof:  The  argument  is  almost  identical  for  both  bl  and  b2;  we  will  use  bl  for  illustra- 
tion. By  axiom  3.13,  it  suffices  to  show  that  each  layout  scene(bl,7)  is  physically  possible. 
By  axiom  2.2,    any  time  T  is  either  before  or  after  ta.  By  lemma  5.9.  for  7"sta.  scenefbl.T) 
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=  scene(bO,r).  and  by  axioms  3.13  and  3.14  the  layouts  of  bO  are  physically  possible.  To 
show  that  scene(bl,r)  is  physically  possible,  it  suffices,  by  axiom  3.12,  to  show  that  the 
places  of  no  rwo  objects  overlap  in  the  layout.  If  neither  of  the  two  objects  is  equal  to  omys- 
rery.  then,  by  lemma  5.9,  their  places  in  bl  are  the  same  as  in  bO.  Since,  by  axioms  3.12, 
3.13,  and  3.14,  they  do  not  overlap  in  bO,  they  cannot  overlap  in  bl. 

The  remaining  case  to  consider  is  to  rule  out  the  possibility  that  omystery  overlaps  some 
other  object,  say  ol,  after  time  ta.  By  lemma  5.9  for  bl,  and  by  lemma  5.10  for  b2,  after 
time  ta.  omvstery  is  strictly  inside  the  region  xenvelope;  hence,  by  definition  1.11,  the  place 
of  omystery  is  a  subset  of  xenvelope.  By  hypothesis  5.6,  ol  never  occupies  a  place  overlap- 
ping xenvelope.  Hence,  by  lemma  1.9,  the  place  of  ol  never  overlaps  the  place  of  omystery. 

Lemma  5.13:  In  each  of  the  behaviors  bO,  bl,  and  b2,  at  all  times  during  the  interval  iO, 
the  object  omystery  is  invisible  to  claire. 

[  [  fl  =  bl  V  S  =  b2  V  fi  =  b3  ]   •  L  =  scene(fi  ,7")  ]  => 
wholly_invis!ble(omystery,  aclaire,  Z, ) 

Proof:  From  hypotheses  5.3,  5.4,  and  5  5  for  bO.  lemma  5.9  for  bl,  and  lemma  5.10  for 
b2.  the  mvstery  object  remains  in  the  point  set  xenvelope  throughout  iO;  Claire  remains  in 
xclaire:  and  the  wall  remains  filling  xwall.  By  hypothesis  5.7,  xwall  blocks  xenvelope  from 
xclaire.  Bv  lemma  1.17,  therefore,  the  place  of  the  mystery  object  is  alwavs  blocked  by  xwall 
from  xclaire.  Hence,  using  definitions  3.7  and  3.10  and  hypothesis  5.1,  the  mystery  object  is 
always  invisible  to  Claire. 

Lemma  5.14:  The  behaviors  bl  and  b2  are  each  visually  compatible  with  Claire's  per- 
ceptions in  bO  up  to  the  end  of  lU. 

bv_compatible { aclaire, behavior(C  )  .bl  ,tz)   •   hv_compatible( aclaire, behavior(r  )  ,b2,tz) 

Proof:  W  all  times  up  to  the  start  of  lO,  bl  and  b2  are  identical  to  bO,  by  lemmas  5.9 
and  5.10.  During  iO,  by  lemmas  5.9  and  5.10.  the  only  object  which  may  be  in  a  different 
position  in  either  bl  or  b2  from  its  position  in  bO  is  omystery,  and  by  lemma  5.13,  it  is 
always  wholly  invisible  to  Claire  throughout  bO,  bl,  and  b2.  .Moreover,  by  lemma  5.12,  bl 
and  b2  are  physically  possible.  Hence,  from  lemma  4.4,  we  may  conclude  that  bl  and  b2  are 
visually  compatible  with  bO  up  to  time  tz. 

Lemma  5.15:  The  behavior  bl  is  visually  compatible  with  Claire's  perceptions  up  to  the 

end  of  lO,  and  omystery  is  motionless  in  bl  during  lO. 

bv_compatible(aclaire,behavior(C),bl,end(iO))  /  motionless(omystery ,bl,iO)  ] 

Proof:  Immediate  from  lemmas  5.9  and  5.14 

Lemma  5.16:  The  behavior  b2  is  visually  compatible  with  Claire's  perceptions  up  to  the 
end  of  iO,  and  omystery  is  not  motionless  in  b2  during  lO. 

bv_compatible(aclaire,behavior(C),bl,tz)  '  -motionless(omystery,bl,iO)  ] 

Proof:  Immediate  from  lemmas  5.10  and  5.14 

Theorem  5.17:  Claire  does  not  know,  at  the  end  of  iO,  whether  or  not  omystery  has 
been  motionless  during  iO. 

[  3(-,  k(aciaire,sOz,  situation(Cl,tz))  A  motionless(omystery,behavior(Cl),iO)  ]  a 
[  3^1  k(aclaire,s0z,siruation(C2,tz))  a  -motionless(omystery,behavior(C2),iO)  J 

Proof:  Immediate  from  Axiom  4.14  and  Lemmas  5.15  and  5.17. 
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Figure  1: 
O  occludes  P  from  A 


Enclosure 


Figure  3: 

Example  III: 

If  Claire  goes  to  the  room 
she  will  see  whether 
the  cat  is  there 
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Figure  2: 

Example  I: 

Claire  cannot  know 
whether  the  object  is  moving 


(sO   )  >  (sia J)         (sihj 


T=9;00  (S2aj         fs2bj 

In  S2a  and  S2b,  flower  inside  vase 


"^         Precedes  relation 
j>        Knowledge  accessible 


Sla.Slb  —  instances  of  SI 


Figure  4: 

Mark  knows  the  flower  was 
in  the  vase  at  9:00 


If  these  relations  hold 


vSOB 
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then  the  diagram  can  be  completed 
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Figure  5: 
Axiom  of  Memory 
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